2024-09-05T01:45:44.896 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-05T01:45:44.922 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T01:45:45.017 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7889793' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1743 sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 sleep_before_teardown: 0 subset: 459/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrzndYWpuRY+iocPai9zQKj9QAi9W0N0DnQfwRX/URzBNkgMS7FR6T0Mcfeb/WSEX9SO7Oil8UZBtO+UYzBUVw= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHugPCQ3r5Lk2JxSNELpPo1Ky2kxR5zDSt/LoMz8cJDCJYMNhMNjrs+hgWTHMTOIUtomLBdf6PagAk41pFQGrbY= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEb/WzC3NBUa+PAzyh2gj0enAEabdxyGxeuGVb+EQji+YvLyGNibBaTxRHRBlBiByYXZEf+k/fA9GN4g6WnIQWg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-04_21:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-05T01:45:45.017 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa; will attempt to use it 2024-09-05T01:45:45.018 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa/tasks 2024-09-05T01:45:45.018 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-05T01:45:45.020 INFO:teuthology.task.internal:Checking packages... 2024-09-05T01:45:45.041 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4' 2024-09-05T01:45:45.041 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-05T01:45:45.041 INFO:teuthology.packaging:ref: None 2024-09-05T01:45:45.042 INFO:teuthology.packaging:tag: None 2024-09-05T01:45:45.042 INFO:teuthology.packaging:branch: squid 2024-09-05T01:45:45.042 INFO:teuthology.packaging:sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:45:45.042 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-05T01:45:45.235 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-259.g91c8a7e5 2024-09-05T01:45:45.236 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-05T01:45:45.241 INFO:teuthology.task.internal:no buildpackages task found 2024-09-05T01:45:45.241 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-05T01:45:45.270 INFO:teuthology.task.internal:Saving configuration 2024-09-05T01:45:45.288 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-05T01:45:45.319 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-05T01:45:45.344 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 01:40:20.577567', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrzndYWpuRY+iocPai9zQKj9QAi9W0N0DnQfwRX/URzBNkgMS7FR6T0Mcfeb/WSEX9SO7Oil8UZBtO+UYzBUVw='} 2024-09-05T01:45:45.365 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 01:40:20.576339', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHugPCQ3r5Lk2JxSNELpPo1Ky2kxR5zDSt/LoMz8cJDCJYMNhMNjrs+hgWTHMTOIUtomLBdf6PagAk41pFQGrbY='} 2024-09-05T01:45:45.386 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 01:40:20.574939', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEb/WzC3NBUa+PAzyh2gj0enAEabdxyGxeuGVb+EQji+YvLyGNibBaTxRHRBlBiByYXZEf+k/fA9GN4g6WnIQWg='} 2024-09-05T01:45:45.387 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-05T01:45:45.392 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-05T01:45:45.393 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-05T01:45:45.393 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-05T01:45:45.393 INFO:teuthology.run_tasks:Running task console_log... 2024-09-05T01:45:45.501 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdb745f7be0>, signals=[15]) 2024-09-05T01:45:45.502 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-05T01:45:45.508 INFO:teuthology.task.internal:Opening connections... 2024-09-05T01:45:45.508 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-09-05T01:45:45.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T01:45:45.589 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-09-05T01:45:45.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T01:45:45.667 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-09-05T01:45:45.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T01:45:45.749 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-05T01:45:45.756 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-09-05T01:45:45.774 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-09-05T01:45:45.774 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-09-05T01:45:45.832 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-09-05T01:45:45.832 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-09-05T01:45:45.832 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-09-05T01:45:45.832 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-09-05T01:45:45.832 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-09-05T01:45:45.832 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-09-05T01:45:45.832 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T01:45:45.832 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-09-05T01:45:45.833 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-09-05T01:45:45.833 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T01:45:45.833 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-09-05T01:45:45.833 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T01:45:45.833 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T01:45:45.833 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T01:45:45.833 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-09-05T01:45:45.862 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-09-05T01:45:45.880 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-09-05T01:45:45.880 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-09-05T01:45:45.938 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-09-05T01:45:45.938 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-09-05T01:45:45.938 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-09-05T01:45:45.938 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-09-05T01:45:45.938 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-09-05T01:45:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-09-05T01:45:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T01:45:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-09-05T01:45:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-09-05T01:45:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T01:45:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-09-05T01:45:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T01:45:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T01:45:45.939 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T01:45:45.939 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-09-05T01:45:45.962 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-09-05T01:45:45.981 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-09-05T01:45:45.981 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-09-05T01:45:46.039 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T01:45:46.040 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T01:45:46.041 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T01:45:46.041 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-09-05T01:45:46.062 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-05T01:45:46.072 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-05T01:45:46.127 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-05T01:45:46.128 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-09-05T01:45:46.130 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-09-05T01:45:46.133 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-09-05T01:45:46.154 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-05T01:45:46.159 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-05T01:45:46.160 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-09-05T01:45:46.188 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-09-05T01:45:46.190 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-09-05T01:45:46.288 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-05T01:45:46.361 INFO:teuthology.run_tasks:Running task kernel... 2024-09-05T01:45:46.375 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-05T01:45:46.375 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-05T01:45:46.376 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T01:45:46.376 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T01:45:46.377 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T01:45:46.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:45:46.394 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-05T01:45:46.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:45:46.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:45:46.396 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-09-05T01:45:46.397 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-05T01:45:46.454 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-503.el9.x86_64 2024-09-05T01:45:46.454 INFO:teuthology.task.kernel:Running kernel on smithi103: 5.14.0-503.el9.x86_64 2024-09-05T01:45:46.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum install -y kernel 2024-09-05T01:45:46.456 INFO:teuthology.orchestra.run.smithi167.stdout:5.14.0-503.el9.x86_64 2024-09-05T01:45:46.456 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-503.el9.x86_64 2024-09-05T01:45:46.457 INFO:teuthology.task.kernel:Running kernel on smithi167: 5.14.0-503.el9.x86_64 2024-09-05T01:45:46.457 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum install -y kernel 2024-09-05T01:45:46.457 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-503.el9.x86_64 2024-09-05T01:45:46.457 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-09-05T01:45:49.031 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-05T01:45:50.185 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-05T01:45:50.218 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-05T01:45:50.832 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-05T01:45:52.102 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-05T01:45:53.919 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 20 MB 00:02 2024-09-05T01:45:55.108 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-05T01:45:56.592 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 19 kB 00:00 2024-09-05T01:45:56.787 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.5 MB 00:00 2024-09-05T01:45:57.479 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-05T01:45:58.144 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-05T01:45:59.204 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-05T01:46:00.353 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 19 kB 00:00 2024-09-05T01:46:00.392 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-05T01:46:01.216 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-05T01:46:02.919 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-05T01:46:04.900 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T01:46:04.900 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T01:46:04.976 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-09-05T01:46:04.977 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-09-05T01:46:04.977 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-09-05T01:46:05.106 DEBUG:teuthology.orchestra.run.smithi167:> echo no | sudo yum reinstall kernel || true 2024-09-05T01:46:05.686 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 01:46:02 AM UTC. 2024-09-05T01:46:05.830 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-09-05T01:46:05.839 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout:Reinstalling: 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 1.9 M 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 0 2024-09-05T01:46:05.840 INFO:teuthology.orchestra.run.smithi167.stderr:Operation aborted. 2024-09-05T01:46:05.904 INFO:teuthology.orchestra.run.smithi167.stdout:Is this ok [y/N]: 2024-09-05T01:46:05.904 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum reinstall -y kernel || true 2024-09-05T01:46:05.931 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-05T01:46:06.445 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:04 ago on Thu 05 Sep 2024 01:46:02 AM UTC. 2024-09-05T01:46:06.571 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-09-05T01:46:06.572 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-09-05T01:46:06.572 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2024-09-05T01:46:06.572 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-09-05T01:46:06.573 INFO:teuthology.orchestra.run.smithi167.stdout:Reinstalling: 2024-09-05T01:46:06.573 INFO:teuthology.orchestra.run.smithi167.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T01:46:06.573 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:46:06.573 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-09-05T01:46:06.573 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-09-05T01:46:06.573 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:46:06.573 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 1.9 M 2024-09-05T01:46:06.573 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 0 2024-09-05T01:46:06.573 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-09-05T01:46:06.730 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-05T01:46:07.271 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-5.14.0-503.el9.x86_64.rpm 2.8 MB/s | 1.9 MB 00:00 2024-09-05T01:46:07.271 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:46:07.271 INFO:teuthology.orchestra.run.smithi167.stdout:Total 2.7 MB/s | 1.9 MB 00:00 2024-09-05T01:46:07.307 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-09-05T01:46:07.525 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-09-05T01:46:07.526 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-09-05T01:46:07.557 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-09-05T01:46:07.557 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-09-05T01:46:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-09-05T01:46:07.823 INFO:teuthology.orchestra.run.smithi167.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T01:46:07.823 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T01:46:07.921 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T01:46:07.921 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T01:46:07.998 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-05T01:46:07.999 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-05T01:46:07.999 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-05T01:46:08.114 DEBUG:teuthology.orchestra.run.smithi103:> echo no | sudo yum reinstall kernel || true 2024-09-05T01:46:08.539 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T01:46:08.539 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T01:46:08.693 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 01:46:05 AM UTC. 2024-09-05T01:46:08.703 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T01:46:08.703 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T01:46:08.782 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-05T01:46:08.783 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-05T01:46:08.783 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-05T01:46:08.828 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-05T01:46:08.833 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-05T01:46:08.833 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-05T01:46:08.833 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-05T01:46:08.833 INFO:teuthology.orchestra.run.smithi103.stdout:Reinstalling: 2024-09-05T01:46:08.833 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T01:46:08.833 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:46:08.833 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-05T01:46:08.833 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-05T01:46:08.833 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:46:08.834 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 1.9 M 2024-09-05T01:46:08.834 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 0 2024-09-05T01:46:08.834 INFO:teuthology.orchestra.run.smithi103.stderr:Operation aborted. 2024-09-05T01:46:08.889 DEBUG:teuthology.orchestra.run.smithi139:> echo no | sudo yum reinstall kernel || true 2024-09-05T01:46:08.896 INFO:teuthology.orchestra.run.smithi103.stdout:Is this ok [y/N]: 2024-09-05T01:46:08.896 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum reinstall -y kernel || true 2024-09-05T01:46:09.266 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T01:46:09.266 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:46:09.266 INFO:teuthology.orchestra.run.smithi167.stdout:Reinstalled: 2024-09-05T01:46:09.266 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T01:46:09.266 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:46:09.266 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-09-05T01:46:09.426 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 01:46:06 AM UTC. 2024-09-05T01:46:09.429 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:04 ago on Thu 05 Sep 2024 01:46:05 AM UTC. 2024-09-05T01:46:09.459 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T01:46:09.518 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T01:46:09.519 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T01:46:09.519 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T01:46:09.519 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-05T01:46:09.519 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T01:46:09.556 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-05T01:46:09.557 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-05T01:46:09.557 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout:Reinstalling: 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 1.9 M 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 0 2024-09-05T01:46:09.558 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-05T01:46:09.575 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2024-09-05T01:46:09.586 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-05T01:46:09.594 DEBUG:teuthology.parallel:result is None 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout:Reinstalling: 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 1.9 M 2024-09-05T01:46:09.596 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 0 2024-09-05T01:46:09.597 INFO:teuthology.orchestra.run.smithi139.stderr:Operation aborted. 2024-09-05T01:46:09.650 INFO:teuthology.orchestra.run.smithi139.stdout:Is this ok [y/N]: 2024-09-05T01:46:09.650 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum reinstall -y kernel || true 2024-09-05T01:46:10.183 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:04 ago on Thu 05 Sep 2024 01:46:06 AM UTC. 2024-09-05T01:46:10.200 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-503.el9.x86_64.rpm 4.8 MB/s | 1.9 MB 00:00 2024-09-05T01:46:10.200 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:46:10.200 INFO:teuthology.orchestra.run.smithi103.stdout:Total 2.9 MB/s | 1.9 MB 00:00 2024-09-05T01:46:10.229 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-05T01:46:10.310 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout:Reinstalling: 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:46:10.311 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 1.9 M 2024-09-05T01:46:10.312 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 0 2024-09-05T01:46:10.312 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-05T01:46:10.459 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-05T01:46:10.459 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-05T01:46:10.489 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-05T01:46:10.489 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-05T01:46:10.590 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-503.el9.x86_64.rpm 8.2 MB/s | 1.9 MB 00:00 2024-09-05T01:46:10.590 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:46:10.590 INFO:teuthology.orchestra.run.smithi139.stdout:Total 6.7 MB/s | 1.9 MB 00:00 2024-09-05T01:46:10.624 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-05T01:46:10.685 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-05T01:46:10.777 INFO:teuthology.orchestra.run.smithi103.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T01:46:10.778 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T01:46:10.861 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-05T01:46:10.861 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-05T01:46:10.891 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-05T01:46:10.892 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-05T01:46:11.045 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-05T01:46:11.151 INFO:teuthology.orchestra.run.smithi139.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T01:46:11.151 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T01:46:11.572 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T01:46:11.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T01:46:11.914 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T01:46:11.914 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T01:46:12.322 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T01:46:12.322 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:46:12.322 INFO:teuthology.orchestra.run.smithi103.stdout:Reinstalled: 2024-09-05T01:46:12.322 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T01:46:12.322 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:46:12.323 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-05T01:46:12.499 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T01:46:12.557 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T01:46:12.557 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T01:46:12.557 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T01:46:12.557 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-05T01:46:12.558 DEBUG:teuthology.orchestra.run.smithi103:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T01:46:12.588 INFO:teuthology.orchestra.run.smithi103.stdout:ttyS1 2024-09-05T01:46:12.604 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T01:46:12.604 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:46:12.604 INFO:teuthology.orchestra.run.smithi139.stdout:Reinstalled: 2024-09-05T01:46:12.604 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T01:46:12.604 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:46:12.604 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-05T01:46:12.609 DEBUG:teuthology.parallel:result is None 2024-09-05T01:46:12.787 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T01:46:12.846 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T01:46:12.846 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T01:46:12.846 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T01:46:12.846 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-05T01:46:12.846 DEBUG:teuthology.orchestra.run.smithi139:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T01:46:12.877 INFO:teuthology.orchestra.run.smithi139.stdout:ttyS1 2024-09-05T01:46:12.904 DEBUG:teuthology.parallel:result is None 2024-09-05T01:46:12.904 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-05T01:46:12.911 INFO:teuthology.task.internal:Creating test directory... 2024-09-05T01:46:12.911 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T01:46:12.914 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T01:46:12.947 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T01:46:12.966 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-05T01:46:13.002 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-05T01:46:13.076 INFO:teuthology.task.internal:Creating archive directory... 2024-09-05T01:46:13.076 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T01:46:13.078 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T01:46:13.081 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T01:46:13.124 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-05T01:46:13.136 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-05T01:46:13.136 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T01:46:13.148 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T01:46:13.151 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T01:46:13.190 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T01:46:13.192 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T01:46:13.196 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T01:46:13.200 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T01:46:13.202 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T01:46:13.207 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T01:46:13.209 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-05T01:46:13.216 INFO:teuthology.task.internal:Configuring sudo... 2024-09-05T01:46:13.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T01:46:13.218 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T01:46:13.246 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T01:46:13.276 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-05T01:46:13.285 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-05T01:46:13.285 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T01:46:13.288 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T01:46:13.311 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T01:46:13.333 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T01:46:13.384 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T01:46:13.437 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T01:46:13.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T01:46:13.560 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:46:13.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T01:46:13.625 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T01:46:13.667 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T01:46:13.712 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T01:46:13.793 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T01:46:13.833 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:46:13.833 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T01:46:13.898 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T01:46:13.945 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T01:46:14.000 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T01:46:14.084 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T01:46:14.127 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:46:14.128 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T01:46:14.195 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-09-05T01:46:14.198 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-09-05T01:46:14.200 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-09-05T01:46:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T01:46:14.231 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T01:46:14.272 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T01:46:14.685 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-05T01:46:14.693 INFO:teuthology.task.internal:Starting timer... 2024-09-05T01:46:14.693 INFO:teuthology.run_tasks:Running task pcp... 2024-09-05T01:46:14.745 INFO:teuthology.run_tasks:Running task selinux... 2024-09-05T01:46:14.756 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-05T01:46:14.756 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-09-05T01:46:14.810 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-09-05T01:46:14.811 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-09-05T01:46:14.854 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-09-05T01:46:14.856 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-09-05T01:46:14.908 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-09-05T01:46:14.910 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-05T01:46:14.910 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-09-05T01:46:14.935 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-09-05T01:46:14.936 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-09-05T01:46:14.961 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-09-05T01:46:14.961 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-09-05T01:46:14.986 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-09-05T01:46:14.986 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-09-05T01:46:14.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T01:46:15.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:46:15.013 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T01:46:15.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:46:15.040 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T01:46:15.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:46:15.066 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-05T01:46:15.066 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-09-05T01:46:15.094 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-09-05T01:46:15.131 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-09-05T01:46:15.164 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-05T01:46:15.174 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-05T01:46:15.186 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-05T01:46:15.187 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi103.front.sepia.ceph.com,smithi139.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-05T01:49:56.362 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-09-05T01:49:56.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-05T01:49:56.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T01:49:56.441 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-05T01:49:56.520 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-05T01:49:56.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-05T01:49:56.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T01:49:56.596 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-05T01:49:56.672 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-05T01:49:56.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-05T01:49:56.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T01:49:56.750 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-09-05T01:49:56.833 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-05T01:49:56.834 INFO:teuthology.run_tasks:Running task clock... 2024-09-05T01:49:56.843 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-05T01:49:56.844 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T01:49:56.844 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T01:49:56.847 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T01:49:56.847 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T01:49:56.849 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T01:49:56.849 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T01:49:56.874 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T01:49:56.877 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T01:49:56.889 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T01:49:56.891 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T01:49:56.904 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T01:49:56.913 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-09-05T01:49:56.915 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-09-05T01:49:56.920 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T01:49:56.925 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-09-05T01:49:56.926 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-09-05T01:49:56.938 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T01:49:56.940 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T01:49:56.945 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-09-05T01:49:56.952 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T01:49:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T01:49:56.956 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-09-05T01:49:56.971 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T01:49:56.985 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T01:49:57.016 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-05T01:49:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-05T01:49:57.020 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T01:49:57.020 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-05T01:49:57.020 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.020 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.021 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.021 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.021 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T01:49:57.022 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-05T01:49:57.022 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.022 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.022 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.022 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.039 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-09-05T01:49:57.042 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T01:49:57.042 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-09-05T01:49:57.042 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.042 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.042 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.043 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:49:57.043 INFO:teuthology.run_tasks:Running task install... 2024-09-05T01:49:57.052 DEBUG:teuthology.task.install:project ceph 2024-09-05T01:49:57.052 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4'}} 2024-09-05T01:49:57.052 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4'} 2024-09-05T01:49:57.052 INFO:teuthology.task.install:Using flavor: default 2024-09-05T01:49:57.057 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-05T01:49:57.057 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-05T01:49:57.058 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'tag': None, 'wait_for_package': False} 2024-09-05T01:49:57.058 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:49:57.060 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'tag': None, 'wait_for_package': False} 2024-09-05T01:49:57.060 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:49:57.061 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'tag': None, 'wait_for_package': False} 2024-09-05T01:49:57.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:49:57.203 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/ 2024-09-05T01:49:57.203 INFO:teuthology.task.install.rpm:Package version is 19.1.1-259.g91c8a7e5 2024-09-05T01:49:57.211 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/ 2024-09-05T01:49:57.211 INFO:teuthology.task.install.rpm:Package version is 19.1.1-259.g91c8a7e5 2024-09-05T01:49:57.214 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/ 2024-09-05T01:49:57.214 INFO:teuthology.task.install.rpm:Package version is 19.1.1-259.g91c8a7e5 2024-09-05T01:49:57.320 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T01:49:57.320 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:49:57.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T01:49:57.325 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T01:49:57.325 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:49:57.325 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T01:49:57.329 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T01:49:57.329 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:49:57.329 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T01:49:57.348 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T01:49:57.348 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T01:49:57.351 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T01:49:57.352 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T01:49:57.356 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T01:49:57.356 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T01:49:57.415 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T01:49:57.417 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T01:49:57.421 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T01:49:57.490 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T01:49:57.492 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T01:49:57.496 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T01:49:57.555 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-09-05T01:49:57.556 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-09-05T01:49:57.557 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-09-05T01:49:57.558 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-09-05T01:49:57.561 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-09-05T01:49:57.562 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-09-05T01:49:57.781 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-09-05T01:49:57.783 INFO:teuthology.orchestra.run.smithi139.stdout:45 files removed 2024-09-05T01:49:57.787 INFO:teuthology.orchestra.run.smithi167.stdout:45 files removed 2024-09-05T01:49:57.807 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T01:49:57.810 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T01:49:57.813 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T01:49:58.352 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 316 kB/s | 84 kB 00:00 2024-09-05T01:49:58.363 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-05T01:49:58.368 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 292 kB/s | 84 kB 00:00 2024-09-05T01:49:58.635 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-05T01:49:58.643 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-05T01:49:58.647 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-05T01:49:58.911 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T01:49:58.914 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-05T01:49:58.920 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T01:49:59.615 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-05T01:50:00.736 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-05T01:50:00.787 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-05T01:50:01.408 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-05T01:50:05.876 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-05T01:50:06.370 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-05T01:50:07.310 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-05T01:50:08.240 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-05T01:50:12.695 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-05T01:50:13.671 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-05T01:50:14.171 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-05T01:50:14.883 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 38 MB/s | 23 MB 00:00 2024-09-05T01:50:15.933 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:50:15.934 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:50:15.947 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T01:50:15.948 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T01:50:15.949 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T01:50:15.950 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T01:50:15.952 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T01:50:15.955 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T01:50:15.956 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T01:50:16.018 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-09-05T01:50:16.027 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================== 2024-09-05T01:50:16.027 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-09-05T01:50:16.027 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================== 2024-09-05T01:50:16.027 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-09-05T01:50:16.027 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T01:50:16.027 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 6.6 k 2024-09-05T01:50:16.027 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.4 M 2024-09-05T01:50:16.027 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 896 k 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 145 k 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 1.5 M 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 146 k 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 3.6 M 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 7.4 M 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 49 k 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 11 M 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 49 M 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 285 k 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 765 k 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 224 k 2024-09-05T01:50:16.028 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 27 k 2024-09-05T01:50:16.029 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 18 k 2024-09-05T01:50:16.029 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T01:50:16.030 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T01:50:16.030 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T01:50:16.030 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 32 k 2024-09-05T01:50:16.030 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 752 k 2024-09-05T01:50:16.032 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 127 k 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 162 k 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 321 k 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 301 k 2024-09-05T01:50:16.033 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 99 k 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 85 k 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.1 M 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 170 k 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T01:50:16.034 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.4 M 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.2 M 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 21 M 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 32 k 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 2.4 M 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 249 k 2024-09-05T01:50:16.035 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 4.7 M 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 17 M 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 16 k 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 25 k 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T01:50:16.036 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 163 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 506 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.3 M 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T01:50:16.037 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T01:50:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 45 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 136 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T01:50:16.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T01:50:16.040 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T01:50:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T01:50:16.042 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T01:50:16.043 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================== 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 6 Packages 2024-09-05T01:50:16.044 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:50:16.045 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 222 M 2024-09-05T01:50:16.045 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-09-05T01:50:16.556 INFO:teuthology.orchestra.run.smithi167.stdout:(1/154): ceph-19.1.1-259.g91c8a7e5.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-05T01:50:16.864 INFO:teuthology.orchestra.run.smithi167.stdout:(2/154): ceph-base-19.1.1-259.g91c8a7e5.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-05T01:50:16.889 INFO:teuthology.orchestra.run.smithi167.stdout:(3/154): ceph-fuse-19.1.1-259.g91c8a7e5.el9.x86 2.6 MB/s | 896 kB 00:00 2024-09-05T01:50:16.932 INFO:teuthology.orchestra.run.smithi167.stdout:(4/154): ceph-immutable-object-cache-19.1.1-259 2.1 MB/s | 145 kB 00:00 2024-09-05T01:50:17.032 INFO:teuthology.orchestra.run.smithi167.stdout:(5/154): ceph-mgr-19.1.1-259.g91c8a7e5.el9.x86_ 15 MB/s | 1.5 MB 00:00 2024-09-05T01:50:17.257 INFO:teuthology.orchestra.run.smithi167.stdout:(6/154): ceph-mon-19.1.1-259.g91c8a7e5.el9.x86_ 21 MB/s | 4.7 MB 00:00 2024-09-05T01:50:17.307 INFO:teuthology.orchestra.run.smithi167.stdout:(7/154): ceph-mds-19.1.1-259.g91c8a7e5.el9.x86_ 5.7 MB/s | 2.4 MB 00:00 2024-09-05T01:50:17.716 INFO:teuthology.orchestra.run.smithi167.stdout:(8/154): ceph-common-19.1.1-259.g91c8a7e5.el9.x 16 MB/s | 21 MB 00:01 2024-09-05T01:50:17.766 INFO:teuthology.orchestra.run.smithi167.stdout:(9/154): ceph-selinux-19.1.1-259.g91c8a7e5.el9. 507 kB/s | 25 kB 00:00 2024-09-05T01:50:18.050 INFO:teuthology.orchestra.run.smithi167.stdout:(10/154): ceph-osd-19.1.1-259.g91c8a7e5.el9.x86 21 MB/s | 17 MB 00:00 2024-09-05T01:50:18.100 INFO:teuthology.orchestra.run.smithi167.stdout:(11/154): cephfs-mirror-19.1.1-259.g91c8a7e5.el 4.4 MB/s | 224 kB 00:00 2024-09-05T01:50:18.167 INFO:teuthology.orchestra.run.smithi167.stdout:(12/154): cephfs-shell-19.1.1-259.g91c8a7e5.el9 406 kB/s | 27 kB 00:00 2024-09-05T01:50:18.226 INFO:teuthology.orchestra.run.smithi167.stdout:(13/154): libcephfs-devel-19.1.1-259.g91c8a7e5. 542 kB/s | 32 kB 00:00 2024-09-05T01:50:18.301 INFO:teuthology.orchestra.run.smithi167.stdout:(14/154): libcephfs2-19.1.1-259.g91c8a7e5.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-05T01:50:18.368 INFO:teuthology.orchestra.run.smithi167.stdout:(15/154): libcephsqlite-19.1.1-259.g91c8a7e5.el 2.4 MB/s | 163 kB 00:00 2024-09-05T01:50:18.435 INFO:teuthology.orchestra.run.smithi167.stdout:(16/154): librados-devel-19.1.1-259.g91c8a7e5.e 1.9 MB/s | 127 kB 00:00 2024-09-05T01:50:18.510 INFO:teuthology.orchestra.run.smithi167.stdout:(17/154): libradosstriper1-19.1.1-259.g91c8a7e5 6.6 MB/s | 506 kB 00:00 2024-09-05T01:50:18.627 INFO:teuthology.orchestra.run.smithi167.stdout:(18/154): ceph-radosgw-19.1.1-259.g91c8a7e5.el9 8.0 MB/s | 11 MB 00:01 2024-09-05T01:50:18.669 INFO:teuthology.orchestra.run.smithi167.stdout:(19/154): python3-ceph-argparse-19.1.1-259.g91c 1.1 MB/s | 45 kB 00:00 2024-09-05T01:50:18.803 INFO:teuthology.orchestra.run.smithi167.stdout:(20/154): librgw2-19.1.1-259.g91c8a7e5.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-05T01:50:18.837 INFO:teuthology.orchestra.run.smithi167.stdout:(21/154): python3-ceph-common-19.1.1-259.g91c8a 812 kB/s | 136 kB 00:00 2024-09-05T01:50:18.862 INFO:teuthology.orchestra.run.smithi167.stdout:(22/154): python3-cephfs-19.1.1-259.g91c8a7e5.e 2.7 MB/s | 162 kB 00:00 2024-09-05T01:50:18.896 INFO:teuthology.orchestra.run.smithi167.stdout:(23/154): python3-rados-19.1.1-259.g91c8a7e5.el 5.4 MB/s | 321 kB 00:00 2024-09-05T01:50:18.929 INFO:teuthology.orchestra.run.smithi167.stdout:(24/154): python3-rbd-19.1.1-259.g91c8a7e5.el9. 4.4 MB/s | 301 kB 00:00 2024-09-05T01:50:18.954 INFO:teuthology.orchestra.run.smithi167.stdout:(25/154): python3-rgw-19.1.1-259.g91c8a7e5.el9. 1.7 MB/s | 99 kB 00:00 2024-09-05T01:50:18.980 INFO:teuthology.orchestra.run.smithi167.stdout:(26/154): rbd-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-05T01:50:19.039 INFO:teuthology.orchestra.run.smithi167.stdout:(27/154): rbd-nbd-19.1.1-259.g91c8a7e5.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-05T01:50:19.097 INFO:teuthology.orchestra.run.smithi167.stdout:(28/154): ceph-grafana-dashboards-19.1.1-259.g9 548 kB/s | 32 kB 00:00 2024-09-05T01:50:19.156 INFO:teuthology.orchestra.run.smithi167.stdout:(29/154): ceph-mgr-cephadm-19.1.1-259.g91c8a7e5 2.4 MB/s | 146 kB 00:00 2024-09-05T01:50:19.307 INFO:teuthology.orchestra.run.smithi167.stdout:(30/154): rbd-mirror-19.1.1-259.g91c8a7e5.el9.x 8.8 MB/s | 3.1 MB 00:00 2024-09-05T01:50:19.382 INFO:teuthology.orchestra.run.smithi167.stdout:(31/154): ceph-mgr-dashboard-19.1.1-259.g91c8a7 16 MB/s | 3.6 MB 00:00 2024-09-05T01:50:19.441 INFO:teuthology.orchestra.run.smithi167.stdout:(32/154): ceph-mgr-modules-core-19.1.1-259.g91c 4.2 MB/s | 249 kB 00:00 2024-09-05T01:50:19.482 INFO:teuthology.orchestra.run.smithi167.stdout:(33/154): ceph-mgr-rook-19.1.1-259.g91c8a7e5.el 1.2 MB/s | 49 kB 00:00 2024-09-05T01:50:19.579 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:17 2024-09-05T01:50:19.908 INFO:teuthology.orchestra.run.smithi167.stdout:(34/154): ceph-test-19.1.1-259.g91c8a7e5.el9.x8 23 MB/s | 49 MB 00:02 2024-09-05T01:50:19.933 INFO:teuthology.orchestra.run.smithi167.stdout:(35/154): ceph-prometheus-alerts-19.1.1-259.g91 36 kB/s | 16 kB 00:00 2024-09-05T01:50:19.975 INFO:teuthology.orchestra.run.smithi167.stdout:(36/154): ceph-volume-19.1.1-259.g91c8a7e5.el9. 4.2 MB/s | 285 kB 00:00 2024-09-05T01:50:20.034 INFO:teuthology.orchestra.run.smithi167.stdout:(37/154): cephadm-19.1.1-259.g91c8a7e5.el9.noar 7.4 MB/s | 765 kB 00:00 2024-09-05T01:50:20.051 INFO:teuthology.orchestra.run.smithi167.stdout:(38/154): cephfs-top-19.1.1-259.g91c8a7e5.el9.n 242 kB/s | 18 kB 00:00 2024-09-05T01:50:20.201 INFO:teuthology.orchestra.run.smithi167.stdout:(39/154): ceph-mgr-diskprediction-local-19.1.1- 8.3 MB/s | 7.4 MB 00:00 2024-09-05T01:50:20.305 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-05T01:50:20.468 INFO:teuthology.orchestra.run.smithi167.stdout:(40/154): fuse-2.9.9-16.el9.x86_64.rpm 184 kB/s | 80 kB 00:00 2024-09-05T01:50:20.735 INFO:teuthology.orchestra.run.smithi167.stdout:(41/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 950 kB/s | 253 kB 00:00 2024-09-05T01:50:20.777 INFO:teuthology.orchestra.run.smithi167.stdout:(42/154): libquadmath-11.5.0-2.el9.x86_64.rpm 332 kB/s | 191 kB 00:00 2024-09-05T01:50:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:(43/154): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 106 kB 00:00 2024-09-05T01:50:21.061 INFO:teuthology.orchestra.run.smithi167.stdout:(44/154): python3-pycparser-2.20-6.el9.noarch.r 773 kB/s | 135 kB 00:00 2024-09-05T01:50:21.086 INFO:teuthology.orchestra.run.smithi167.stdout:(45/154): libgfortran-11.5.0-2.el9.x86_64.rpm 774 kB/s | 801 kB 00:01 2024-09-05T01:50:21.178 INFO:teuthology.orchestra.run.smithi167.stdout:(46/154): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-09-05T01:50:21.203 INFO:teuthology.orchestra.run.smithi167.stdout:(47/154): python3-requests-2.25.1-8.el9.noarch. 877 kB/s | 125 kB 00:00 2024-09-05T01:50:21.271 INFO:teuthology.orchestra.run.smithi167.stdout:(48/154): python3-urllib3-1.26.5-6.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-09-05T01:50:21.304 INFO:teuthology.orchestra.run.smithi167.stdout:(49/154): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-09-05T01:50:21.379 INFO:teuthology.orchestra.run.smithi167.stdout:(50/154): boost-program-options-1.75.0-8.el9.x8 986 kB/s | 107 kB 00:00 2024-09-05T01:50:21.405 INFO:teuthology.orchestra.run.smithi167.stdout:(51/154): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-09-05T01:50:21.514 INFO:teuthology.orchestra.run.smithi167.stdout:(52/154): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-05T01:50:21.581 INFO:teuthology.orchestra.run.smithi167.stdout:(53/154): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-09-05T01:50:21.689 INFO:teuthology.orchestra.run.smithi167.stdout:(54/154): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-05T01:50:21.714 INFO:teuthology.orchestra.run.smithi167.stdout:(55/154): flex-2.6.4-9.el9.x86_64.rpm 936 kB/s | 313 kB 00:00 2024-09-05T01:50:21.798 INFO:teuthology.orchestra.run.smithi167.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-05T01:50:21.823 INFO:teuthology.orchestra.run.smithi167.stdout:(57/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-05T01:50:21.870 INFO:teuthology.orchestra.run.smithi167.stdout:(58/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.4 MB/s | 3.0 MB 00:00 2024-09-05T01:50:21.904 INFO:teuthology.orchestra.run.smithi167.stdout:(59/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 428 kB/s | 45 kB 00:00 2024-09-05T01:50:21.988 INFO:teuthology.orchestra.run.smithi167.stdout:(60/154): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2024-09-05T01:50:22.013 INFO:teuthology.orchestra.run.smithi167.stdout:(61/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 225 kB/s | 24 kB 00:00 2024-09-05T01:50:22.097 INFO:teuthology.orchestra.run.smithi167.stdout:(62/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-05T01:50:22.122 INFO:teuthology.orchestra.run.smithi167.stdout:(63/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-05T01:50:22.206 INFO:teuthology.orchestra.run.smithi167.stdout:(64/154): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-05T01:50:22.231 INFO:teuthology.orchestra.run.smithi167.stdout:(65/154): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-05T01:50:22.348 INFO:teuthology.orchestra.run.smithi167.stdout:(66/154): librdkafka-1.6.1-102.el9.x86_64.rpm 1.2 MB/s | 662 kB 00:00 2024-09-05T01:50:22.550 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:50:22.550 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:50:22.561 INFO:teuthology.orchestra.run.smithi139.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T01:50:22.562 INFO:teuthology.orchestra.run.smithi139.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T01:50:22.563 INFO:teuthology.orchestra.run.smithi139.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T01:50:22.564 INFO:teuthology.orchestra.run.smithi139.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T01:50:22.568 INFO:teuthology.orchestra.run.smithi139.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T01:50:22.569 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T01:50:22.569 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T01:50:22.607 INFO:teuthology.orchestra.run.smithi167.stdout:(67/154): openblas-openmp-0.3.26-2.el9.x86_64.r 12 MB/s | 4.9 MB 00:00 2024-09-05T01:50:22.629 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-05T01:50:22.638 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-05T01:50:22.638 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-05T01:50:22.638 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-05T01:50:22.638 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-05T01:50:22.638 INFO:teuthology.orchestra.run.smithi139.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T01:50:22.638 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 6.6 k 2024-09-05T01:50:22.638 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.4 M 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 896 k 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 145 k 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 1.5 M 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 146 k 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 3.6 M 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 7.4 M 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 49 k 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 11 M 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 49 M 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 285 k 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 765 k 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 224 k 2024-09-05T01:50:22.639 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 27 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 18 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 32 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 752 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 127 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T01:50:22.640 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 162 k 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 321 k 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 301 k 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 99 k 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 85 k 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.1 M 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 170 k 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T01:50:22.641 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.4 M 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.2 M 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 21 M 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 32 k 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 2.4 M 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 249 k 2024-09-05T01:50:22.642 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 4.7 M 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 17 M 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 16 k 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 25 k 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 163 k 2024-09-05T01:50:22.643 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 506 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.3 M 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T01:50:22.644 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 45 k 2024-09-05T01:50:22.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 136 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T01:50:22.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T01:50:22.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T01:50:22.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T01:50:22.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T01:50:22.650 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:50:22.651 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-05T01:50:22.651 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-05T01:50:22.651 INFO:teuthology.orchestra.run.smithi139.stdout:Install 148 Packages 2024-09-05T01:50:22.651 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 6 Packages 2024-09-05T01:50:22.651 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:50:22.651 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 222 M 2024-09-05T01:50:22.651 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-05T01:50:22.732 INFO:teuthology.orchestra.run.smithi167.stdout:(68/154): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-05T01:50:22.858 INFO:teuthology.orchestra.run.smithi167.stdout:(69/154): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-05T01:50:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:(70/154): postgresql-server-13.16-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-09-05T01:50:23.415 INFO:teuthology.orchestra.run.smithi139.stdout:(1/154): ceph-19.1.1-259.g91c8a7e5.el9.x86_64.r 52 kB/s | 6.6 kB 00:00 2024-09-05T01:50:23.467 INFO:teuthology.orchestra.run.smithi167.stdout:(71/154): postgresql-13.16-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-09-05T01:50:23.508 INFO:teuthology.orchestra.run.smithi167.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-09-05T01:50:23.575 INFO:teuthology.orchestra.run.smithi167.stdout:(73/154): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-09-05T01:50:23.615 INFO:teuthology.orchestra.run.smithi139.stdout:(2/154): ceph-fuse-19.1.1-259.g91c8a7e5.el9.x86 4.4 MB/s | 896 kB 00:00 2024-09-05T01:50:23.684 INFO:teuthology.orchestra.run.smithi167.stdout:(74/154): python3-colorama-0.4.6-3.el9.noarch.r 566 kB/s | 61 kB 00:00 2024-09-05T01:50:23.724 INFO:teuthology.orchestra.run.smithi139.stdout:(3/154): ceph-immutable-object-cache-19.1.1-259 1.3 MB/s | 145 kB 00:00 2024-09-05T01:50:23.768 INFO:teuthology.orchestra.run.smithi167.stdout:(75/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-09-05T01:50:23.808 INFO:teuthology.orchestra.run.smithi139.stdout:(4/154): ceph-base-19.1.1-259.g91c8a7e5.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-05T01:50:23.885 INFO:teuthology.orchestra.run.smithi167.stdout:(76/154): python3-devel-3.9.19-8.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-09-05T01:50:23.891 INFO:teuthology.orchestra.run.smithi139.stdout:(5/154): ceph-mgr-19.1.1-259.g91c8a7e5.el9.x86_ 18 MB/s | 1.5 MB 00:00 2024-09-05T01:50:24.010 INFO:teuthology.orchestra.run.smithi167.stdout:(77/154): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-05T01:50:24.027 INFO:teuthology.orchestra.run.smithi167.stdout:(78/154): python3-jsonpatch-1.21-16.el9.noarch. 180 kB/s | 26 kB 00:00 2024-09-05T01:50:24.150 INFO:teuthology.orchestra.run.smithi139.stdout:(6/154): ceph-mon-19.1.1-259.g91c8a7e5.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-05T01:50:24.152 INFO:teuthology.orchestra.run.smithi167.stdout:(79/154): python3-jinja2-2.11.3-6.el9.noarch.rp 646 kB/s | 248 kB 00:00 2024-09-05T01:50:24.169 INFO:teuthology.orchestra.run.smithi167.stdout:(80/154): python3-jsonpointer-2.0-4.el9.noarch. 121 kB/s | 19 kB 00:00 2024-09-05T01:50:24.195 INFO:teuthology.orchestra.run.smithi167.stdout:(81/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-05T01:50:24.245 INFO:teuthology.orchestra.run.smithi167.stdout:(82/154): python3-markupsafe-1.1.1-12.el9.x86_6 377 kB/s | 35 kB 00:00 2024-09-05T01:50:24.379 INFO:teuthology.orchestra.run.smithi167.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-09-05T01:50:24.383 INFO:teuthology.orchestra.run.smithi139.stdout:(7/154): ceph-common-19.1.1-259.g91c8a7e5.el9.x 19 MB/s | 21 MB 00:01 2024-09-05T01:50:24.442 INFO:teuthology.orchestra.run.smithi139.stdout:(8/154): ceph-mds-19.1.1-259.g91c8a7e5.el9.x86_ 3.3 MB/s | 2.4 MB 00:00 2024-09-05T01:50:24.487 INFO:teuthology.orchestra.run.smithi167.stdout:(84/154): python3-packaging-20.9-5.el9.noarch.r 712 kB/s | 77 kB 00:00 2024-09-05T01:50:24.509 INFO:teuthology.orchestra.run.smithi139.stdout:(9/154): ceph-selinux-19.1.1-259.g91c8a7e5.el9. 378 kB/s | 25 kB 00:00 2024-09-05T01:50:24.571 INFO:teuthology.orchestra.run.smithi167.stdout:(85/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-05T01:50:24.596 INFO:teuthology.orchestra.run.smithi167.stdout:(86/154): python3-oauthlib-3.1.1-5.el9.noarch.r 633 kB/s | 222 kB 00:00 2024-09-05T01:50:24.621 INFO:teuthology.orchestra.run.smithi167.stdout:(87/154): python3-prettytable-0.7.2-27.el9.noar 315 kB/s | 42 kB 00:00 2024-09-05T01:50:24.680 INFO:teuthology.orchestra.run.smithi167.stdout:(88/154): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-05T01:50:24.730 INFO:teuthology.orchestra.run.smithi167.stdout:(89/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-05T01:50:24.789 INFO:teuthology.orchestra.run.smithi167.stdout:(90/154): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-09-05T01:50:24.818 INFO:teuthology.orchestra.run.smithi139.stdout:(10/154): ceph-radosgw-19.1.1-259.g91c8a7e5.el9 24 MB/s | 11 MB 00:00 2024-09-05T01:50:24.831 INFO:teuthology.orchestra.run.smithi167.stdout:(91/154): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-09-05T01:50:24.893 INFO:teuthology.orchestra.run.smithi139.stdout:(11/154): cephfs-mirror-19.1.1-259.g91c8a7e5.el 2.9 MB/s | 224 kB 00:00 2024-09-05T01:50:24.939 INFO:teuthology.orchestra.run.smithi167.stdout:(92/154): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-05T01:50:24.959 INFO:teuthology.orchestra.run.smithi139.stdout:(12/154): cephfs-shell-19.1.1-259.g91c8a7e5.el9 406 kB/s | 27 kB 00:00 2024-09-05T01:50:25.040 INFO:teuthology.orchestra.run.smithi167.stdout:(93/154): python3-pyasn1-modules-0.4.8-6.el9.no 667 kB/s | 279 kB 00:00 2024-09-05T01:50:25.065 INFO:teuthology.orchestra.run.smithi167.stdout:(94/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 352 kB/s | 44 kB 00:00 2024-09-05T01:50:25.160 INFO:teuthology.orchestra.run.smithi139.stdout:(13/154): ceph-osd-19.1.1-259.g91c8a7e5.el9.x86 17 MB/s | 17 MB 00:01 2024-09-05T01:50:25.174 INFO:teuthology.orchestra.run.smithi167.stdout:(95/154): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-09-05T01:50:25.185 INFO:teuthology.orchestra.run.smithi139.stdout:(14/154): libcephfs-devel-19.1.1-259.g91c8a7e5. 141 kB/s | 32 kB 00:00 2024-09-05T01:50:25.227 INFO:teuthology.orchestra.run.smithi139.stdout:(15/154): libcephfs2-19.1.1-259.g91c8a7e5.el9.x 11 MB/s | 752 kB 00:00 2024-09-05T01:50:25.253 INFO:teuthology.orchestra.run.smithi139.stdout:(16/154): libcephsqlite-19.1.1-259.g91c8a7e5.el 2.4 MB/s | 163 kB 00:00 2024-09-05T01:50:25.274 INFO:teuthology.orchestra.run.smithi167.stdout:(96/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-05T01:50:25.277 INFO:teuthology.orchestra.run.smithi139.stdout:(17/154): librados-devel-19.1.1-259.g91c8a7e5.e 2.5 MB/s | 127 kB 00:00 2024-09-05T01:50:25.303 INFO:teuthology.orchestra.run.smithi139.stdout:(18/154): libradosstriper1-19.1.1-259.g91c8a7e5 9.9 MB/s | 506 kB 00:00 2024-09-05T01:50:25.362 INFO:teuthology.orchestra.run.smithi139.stdout:(19/154): python3-ceph-argparse-19.1.1-259.g91c 775 kB/s | 45 kB 00:00 2024-09-05T01:50:25.383 INFO:teuthology.orchestra.run.smithi167.stdout:(97/154): qatlib-24.02.0-1.el9.x86_64.rpm 644 kB/s | 221 kB 00:00 2024-09-05T01:50:25.429 INFO:teuthology.orchestra.run.smithi139.stdout:(20/154): python3-ceph-common-19.1.1-259.g91c8a 2.0 MB/s | 136 kB 00:00 2024-09-05T01:50:25.458 INFO:teuthology.orchestra.run.smithi167.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-05T01:50:25.483 INFO:teuthology.orchestra.run.smithi167.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 576 kB/s | 58 kB 00:00 2024-09-05T01:50:25.529 INFO:teuthology.orchestra.run.smithi139.stdout:(21/154): librgw2-19.1.1-259.g91c8a7e5.el9.x86_ 21 MB/s | 5.3 MB 00:00 2024-09-05T01:50:25.554 INFO:teuthology.orchestra.run.smithi139.stdout:(22/154): python3-cephfs-19.1.1-259.g91c8a7e5.e 1.3 MB/s | 162 kB 00:00 2024-09-05T01:50:25.567 INFO:teuthology.orchestra.run.smithi167.stdout:(100/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 883 kB/s | 96 kB 00:00 2024-09-05T01:50:25.579 INFO:teuthology.orchestra.run.smithi139.stdout:(23/154): python3-rados-19.1.1-259.g91c8a7e5.el 6.3 MB/s | 321 kB 00:00 2024-09-05T01:50:25.613 INFO:teuthology.orchestra.run.smithi139.stdout:(24/154): python3-rbd-19.1.1-259.g91c8a7e5.el9. 5.0 MB/s | 301 kB 00:00 2024-09-05T01:50:25.638 INFO:teuthology.orchestra.run.smithi139.stdout:(25/154): python3-rgw-19.1.1-259.g91c8a7e5.el9. 1.7 MB/s | 99 kB 00:00 2024-09-05T01:50:25.663 INFO:teuthology.orchestra.run.smithi139.stdout:(26/154): rbd-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-05T01:50:25.667 INFO:teuthology.orchestra.run.smithi167.stdout:(101/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-09-05T01:50:25.692 INFO:teuthology.orchestra.run.smithi167.stdout:(102/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 357 kB/s | 45 kB 00:00 2024-09-05T01:50:25.722 INFO:teuthology.orchestra.run.smithi139.stdout:(27/154): rbd-nbd-19.1.1-259.g91c8a7e5.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-05T01:50:25.776 INFO:teuthology.orchestra.run.smithi167.stdout:(103/154): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-09-05T01:50:25.798 INFO:teuthology.orchestra.run.smithi139.stdout:(28/154): rbd-mirror-19.1.1-259.g91c8a7e5.el9.x 20 MB/s | 3.1 MB 00:00 2024-09-05T01:50:25.823 INFO:teuthology.orchestra.run.smithi139.stdout:(29/154): ceph-grafana-dashboards-19.1.1-259.g9 317 kB/s | 32 kB 00:00 2024-09-05T01:50:25.835 INFO:teuthology.orchestra.run.smithi167.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-05T01:50:25.848 INFO:teuthology.orchestra.run.smithi139.stdout:(30/154): ceph-mgr-cephadm-19.1.1-259.g91c8a7e5 2.8 MB/s | 146 kB 00:00 2024-09-05T01:50:25.893 INFO:teuthology.orchestra.run.smithi167.stdout:(105/154): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-09-05T01:50:25.919 INFO:teuthology.orchestra.run.smithi167.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-05T01:50:25.990 INFO:teuthology.orchestra.run.smithi139.stdout:(31/154): ceph-mgr-dashboard-19.1.1-259.g91c8a7 21 MB/s | 3.6 MB 00:00 2024-09-05T01:50:26.019 INFO:teuthology.orchestra.run.smithi167.stdout:(107/154): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-05T01:50:26.044 INFO:teuthology.orchestra.run.smithi167.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-05T01:50:26.074 INFO:teuthology.orchestra.run.smithi139.stdout:(32/154): ceph-mgr-modules-core-19.1.1-259.g91c 2.9 MB/s | 249 kB 00:00 2024-09-05T01:50:26.125 INFO:teuthology.orchestra.run.smithi139.stdout:(33/154): ceph-mgr-rook-19.1.1-259.g91c8a7e5.el 986 kB/s | 49 kB 00:00 2024-09-05T01:50:26.208 INFO:teuthology.orchestra.run.smithi139.stdout:(34/154): ceph-mgr-diskprediction-local-19.1.1- 21 MB/s | 7.4 MB 00:00 2024-09-05T01:50:26.211 INFO:teuthology.orchestra.run.smithi167.stdout:(109/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-05T01:50:26.225 INFO:teuthology.orchestra.run.smithi139.stdout:(35/154): ceph-prometheus-alerts-19.1.1-259.g91 164 kB/s | 16 kB 00:00 2024-09-05T01:50:26.237 INFO:teuthology.orchestra.run.smithi167.stdout:(110/154): liboath-2.6.7-2.el9.x86_64.rpm 253 kB/s | 49 kB 00:00 2024-09-05T01:50:26.258 INFO:teuthology.orchestra.run.smithi139.stdout:(36/154): ceph-volume-19.1.1-259.g91c8a7e5.el9. 5.6 MB/s | 285 kB 00:00 2024-09-05T01:50:26.270 INFO:teuthology.orchestra.run.smithi167.stdout:(111/154): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-09-05T01:50:26.292 INFO:teuthology.orchestra.run.smithi139.stdout:(37/154): cephadm-19.1.1-259.g91c8a7e5.el9.noar 11 MB/s | 765 kB 00:00 2024-09-05T01:50:26.304 INFO:teuthology.orchestra.run.smithi167.stdout:(112/154): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-09-05T01:50:26.317 INFO:teuthology.orchestra.run.smithi139.stdout:(38/154): cephfs-top-19.1.1-259.g91c8a7e5.el9.n 311 kB/s | 18 kB 00:00 2024-09-05T01:50:26.338 INFO:teuthology.orchestra.run.smithi167.stdout:(113/154): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-09-05T01:50:26.363 INFO:teuthology.orchestra.run.smithi167.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-05T01:50:26.388 INFO:teuthology.orchestra.run.smithi167.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-05T01:50:26.401 INFO:teuthology.orchestra.run.smithi139.stdout:(39/154): fuse-2.9.9-16.el9.x86_64.rpm 735 kB/s | 80 kB 00:00 2024-09-05T01:50:26.413 INFO:teuthology.orchestra.run.smithi167.stdout:(116/154): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-05T01:50:26.455 INFO:teuthology.orchestra.run.smithi167.stdout:(117/154): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-09-05T01:50:26.459 INFO:teuthology.orchestra.run.smithi139.stdout:(40/154): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-05T01:50:26.481 INFO:teuthology.orchestra.run.smithi167.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-05T01:50:26.501 INFO:teuthology.orchestra.run.smithi139.stdout:(41/154): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-05T01:50:26.523 INFO:teuthology.orchestra.run.smithi167.stdout:(119/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.3 MB/s | 839 kB 00:00 2024-09-05T01:50:26.527 INFO:teuthology.orchestra.run.smithi139.stdout:(42/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-05T01:50:26.548 INFO:teuthology.orchestra.run.smithi167.stdout:(120/154): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-05T01:50:26.568 INFO:teuthology.orchestra.run.smithi139.stdout:(43/154): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-05T01:50:26.582 INFO:teuthology.orchestra.run.smithi167.stdout:(121/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-09-05T01:50:26.607 INFO:teuthology.orchestra.run.smithi167.stdout:(122/154): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-05T01:50:26.613 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.5 MB 00:03 2024-09-05T01:50:26.633 INFO:teuthology.orchestra.run.smithi167.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-05T01:50:26.636 INFO:teuthology.orchestra.run.smithi139.stdout:(44/154): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-05T01:50:26.658 INFO:teuthology.orchestra.run.smithi167.stdout:(124/154): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-05T01:50:26.683 INFO:teuthology.orchestra.run.smithi167.stdout:(125/154): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-05T01:50:26.694 INFO:teuthology.orchestra.run.smithi139.stdout:(45/154): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-05T01:50:26.708 INFO:teuthology.orchestra.run.smithi167.stdout:(126/154): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-05T01:50:26.734 INFO:teuthology.orchestra.run.smithi167.stdout:(127/154): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-05T01:50:26.753 INFO:teuthology.orchestra.run.smithi139.stdout:(46/154): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-09-05T01:50:26.759 INFO:teuthology.orchestra.run.smithi167.stdout:(128/154): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-05T01:50:26.786 INFO:teuthology.orchestra.run.smithi139.stdout:(47/154): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-05T01:50:26.801 INFO:teuthology.orchestra.run.smithi167.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-09-05T01:50:26.826 INFO:teuthology.orchestra.run.smithi167.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-05T01:50:26.860 INFO:teuthology.orchestra.run.smithi167.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-05T01:50:26.911 INFO:teuthology.orchestra.run.smithi167.stdout:(132/154): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-05T01:50:26.936 INFO:teuthology.orchestra.run.smithi167.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-05T01:50:26.945 INFO:teuthology.orchestra.run.smithi139.stdout:(48/154): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-09-05T01:50:26.969 INFO:teuthology.orchestra.run.smithi167.stdout:(134/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-09-05T01:50:26.996 INFO:teuthology.orchestra.run.smithi139.stdout:(49/154): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 689 kB 00:00 2024-09-05T01:50:27.011 INFO:teuthology.orchestra.run.smithi167.stdout:(135/154): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-09-05T01:50:27.021 INFO:teuthology.orchestra.run.smithi139.stdout:(50/154): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-05T01:50:27.037 INFO:teuthology.orchestra.run.smithi167.stdout:(136/154): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-09-05T01:50:27.046 INFO:teuthology.orchestra.run.smithi139.stdout:(51/154): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-05T01:50:27.062 INFO:teuthology.orchestra.run.smithi167.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-05T01:50:27.087 INFO:teuthology.orchestra.run.smithi167.stdout:(138/154): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-05T01:50:27.088 INFO:teuthology.orchestra.run.smithi139.stdout:(52/154): flexiblas-3.0.4-8.el9.x86_64.rpm 783 kB/s | 33 kB 00:00 2024-09-05T01:50:27.113 INFO:teuthology.orchestra.run.smithi167.stdout:(139/154): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-05T01:50:27.138 INFO:teuthology.orchestra.run.smithi167.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-05T01:50:27.139 INFO:teuthology.orchestra.run.smithi139.stdout:(53/154): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-05T01:50:27.163 INFO:teuthology.orchestra.run.smithi167.stdout:(141/154): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-05T01:50:27.180 INFO:teuthology.orchestra.run.smithi139.stdout:(54/154): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-05T01:50:27.214 INFO:teuthology.orchestra.run.smithi167.stdout:(142/154): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-09-05T01:50:27.230 INFO:teuthology.orchestra.run.smithi139.stdout:(55/154): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-05T01:50:27.239 INFO:teuthology.orchestra.run.smithi167.stdout:(143/154): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-05T01:50:27.264 INFO:teuthology.orchestra.run.smithi167.stdout:(144/154): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-09-05T01:50:27.290 INFO:teuthology.orchestra.run.smithi167.stdout:(145/154): python3-zc-lockfile-2.0-10.el9.noarc 395 kB/s | 20 kB 00:00 2024-09-05T01:50:27.297 INFO:teuthology.orchestra.run.smithi139.stdout:(56/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-05T01:50:27.315 INFO:teuthology.orchestra.run.smithi167.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-05T01:50:27.341 INFO:teuthology.orchestra.run.smithi167.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-05T01:50:27.347 INFO:teuthology.orchestra.run.smithi139.stdout:(57/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 916 kB/s | 45 kB 00:00 2024-09-05T01:50:27.408 INFO:teuthology.orchestra.run.smithi167.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-05T01:50:27.583 INFO:teuthology.orchestra.run.smithi167.stdout:(149/154): librbd1-19.1.1-259.g91c8a7e5.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-05T01:50:27.683 INFO:teuthology.orchestra.run.smithi167.stdout:(150/154): librados2-19.1.1-259.g91c8a7e5.el9.x 9.3 MB/s | 3.4 MB 00:00 2024-09-05T01:50:28.081 INFO:teuthology.orchestra.run.smithi167.stdout:(151/154): NetworkManager-team-1.48.10-1.el9.x8 211 kB/s | 40 kB 00:00 2024-09-05T01:50:28.081 INFO:teuthology.orchestra.run.smithi167.stdout:(152/154): NetworkManager-1.48.10-1.el9.x86_64. 3.9 MB/s | 2.3 MB 00:00 2024-09-05T01:50:28.081 INFO:teuthology.orchestra.run.smithi139.stdout:(58/154): ceph-test-19.1.1-259.g91c8a7e5.el9.x8 14 MB/s | 49 MB 00:03 2024-09-05T01:50:28.082 INFO:teuthology.orchestra.run.smithi139.stdout:(59/154): librdkafka-1.6.1-102.el9.x86_64.rpm 1.1 MB/s | 662 kB 00:00 2024-09-05T01:50:28.082 INFO:teuthology.orchestra.run.smithi139.stdout:(60/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-09-05T01:50:28.082 INFO:teuthology.orchestra.run.smithi139.stdout:(61/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 244 kB/s | 24 kB 00:00 2024-09-05T01:50:28.082 INFO:teuthology.orchestra.run.smithi139.stdout:(62/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-05T01:50:28.099 INFO:teuthology.orchestra.run.smithi139.stdout:(63/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-05T01:50:28.142 INFO:teuthology.orchestra.run.smithi167.stdout:(153/154): NetworkManager-tui-1.48.10-1.el9.x86 929 kB/s | 248 kB 00:00 2024-09-05T01:50:28.149 INFO:teuthology.orchestra.run.smithi139.stdout:(64/154): libtool-2.4.6-46.el9.x86_64.rpm 2.5 MB/s | 581 kB 00:00 2024-09-05T01:50:28.174 INFO:teuthology.orchestra.run.smithi139.stdout:(65/154): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-05T01:50:28.200 INFO:teuthology.orchestra.run.smithi139.stdout:(66/154): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-05T01:50:28.238 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-05T01:50:28.650 INFO:teuthology.orchestra.run.smithi139.stdout:(67/154): postgresql-13.16-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-09-05T01:50:28.926 INFO:teuthology.orchestra.run.smithi139.stdout:(68/154): postgresql-contrib-13.16-1.el9.x86_64 3.2 MB/s | 910 kB 00:00 2024-09-05T01:50:28.975 INFO:teuthology.orchestra.run.smithi167.stdout:(154/154): NetworkManager-libnm-1.48.10-1.el9.x 1.3 MB/s | 1.8 MB 00:01 2024-09-05T01:50:28.982 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:50:28.982 INFO:teuthology.orchestra.run.smithi167.stdout:Total 17 MB/s | 222 MB 00:12 2024-09-05T01:50:29.001 INFO:teuthology.orchestra.run.smithi139.stdout:(69/154): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-05T01:50:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-05T01:50:29.409 INFO:teuthology.orchestra.run.smithi139.stdout:(70/154): openblas-openmp-0.3.26-2.el9.x86_64.r 3.9 MB/s | 4.9 MB 00:01 2024-09-05T01:50:29.504 INFO:teuthology.orchestra.run.smithi139.stdout:(71/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-09-05T01:50:29.568 INFO:teuthology.orchestra.run.smithi139.stdout:(72/154): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-05T01:50:29.777 INFO:teuthology.orchestra.run.smithi139.stdout:(73/154): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-05T01:50:29.819 INFO:teuthology.orchestra.run.smithi139.stdout:(74/154): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-05T01:50:29.919 INFO:teuthology.orchestra.run.smithi139.stdout:(75/154): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-05T01:50:30.019 INFO:teuthology.orchestra.run.smithi139.stdout:(76/154): postgresql-server-13.16-1.el9.x86_64. 5.7 MB/s | 5.8 MB 00:01 2024-09-05T01:50:30.103 INFO:teuthology.orchestra.run.smithi139.stdout:(77/154): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-05T01:50:30.128 INFO:teuthology.orchestra.run.smithi139.stdout:(78/154): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-09-05T01:50:30.149 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-09-05T01:50:30.153 INFO:teuthology.orchestra.run.smithi139.stdout:(79/154): python3-jinja2-2.11.3-6.el9.noarch.rp 1.0 MB/s | 248 kB 00:00 2024-09-05T01:50:30.178 INFO:teuthology.orchestra.run.smithi139.stdout:(80/154): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-09-05T01:50:30.204 INFO:teuthology.orchestra.run.smithi139.stdout:(81/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-05T01:50:30.229 INFO:teuthology.orchestra.run.smithi139.stdout:(82/154): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-09-05T01:50:30.254 INFO:teuthology.orchestra.run.smithi139.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-05T01:50:30.282 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-09-05T01:50:30.282 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-09-05T01:50:30.288 INFO:teuthology.orchestra.run.smithi139.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-05T01:50:30.313 INFO:teuthology.orchestra.run.smithi139.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-05T01:50:30.347 INFO:teuthology.orchestra.run.smithi139.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 718 kB/s | 42 kB 00:00 2024-09-05T01:50:30.380 INFO:teuthology.orchestra.run.smithi139.stdout:(87/154): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-05T01:50:30.405 INFO:teuthology.orchestra.run.smithi139.stdout:(88/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-05T01:50:30.431 INFO:teuthology.orchestra.run.smithi139.stdout:(89/154): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-05T01:50:30.456 INFO:teuthology.orchestra.run.smithi139.stdout:(90/154): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-05T01:50:30.481 INFO:teuthology.orchestra.run.smithi139.stdout:(91/154): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-05T01:50:30.532 INFO:teuthology.orchestra.run.smithi139.stdout:(92/154): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-05T01:50:30.590 INFO:teuthology.orchestra.run.smithi139.stdout:(93/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 760 kB/s | 44 kB 00:00 2024-09-05T01:50:30.649 INFO:teuthology.orchestra.run.smithi139.stdout:(94/154): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-05T01:50:30.849 INFO:teuthology.orchestra.run.smithi139.stdout:(95/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-09-05T01:50:30.874 INFO:teuthology.orchestra.run.smithi139.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 159 kB/s | 36 kB 00:00 2024-09-05T01:50:30.908 INFO:teuthology.orchestra.run.smithi139.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-09-05T01:50:30.934 INFO:teuthology.orchestra.run.smithi139.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 5.0 MB/s | 304 kB 00:00 2024-09-05T01:50:30.967 INFO:teuthology.orchestra.run.smithi139.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 989 kB/s | 58 kB 00:00 2024-09-05T01:50:31.001 INFO:teuthology.orchestra.run.smithi139.stdout:(100/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-05T01:50:31.026 INFO:teuthology.orchestra.run.smithi139.stdout:(101/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-05T01:50:31.051 INFO:teuthology.orchestra.run.smithi139.stdout:(102/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-05T01:50:31.127 INFO:teuthology.orchestra.run.smithi139.stdout:(103/154): python3-numpy-1.20.1-5.el9.x86_64.rp 5.4 MB/s | 5.1 MB 00:00 2024-09-05T01:50:31.177 INFO:teuthology.orchestra.run.smithi139.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-05T01:50:31.210 INFO:teuthology.orchestra.run.smithi139.stdout:(105/154): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-05T01:50:31.236 INFO:teuthology.orchestra.run.smithi139.stdout:(106/154): lua-devel-5.4.4-4.el9.x86_64.rpm 106 kB/s | 22 kB 00:00 2024-09-05T01:50:31.261 INFO:teuthology.orchestra.run.smithi139.stdout:(107/154): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-09-05T01:50:31.294 INFO:teuthology.orchestra.run.smithi139.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-05T01:50:31.320 INFO:teuthology.orchestra.run.smithi139.stdout:(109/154): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-05T01:50:31.345 INFO:teuthology.orchestra.run.smithi139.stdout:(110/154): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-05T01:50:31.370 INFO:teuthology.orchestra.run.smithi139.stdout:(111/154): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-05T01:50:31.404 INFO:teuthology.orchestra.run.smithi139.stdout:(112/154): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-05T01:50:31.454 INFO:teuthology.orchestra.run.smithi139.stdout:(113/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-05T01:50:31.471 INFO:teuthology.orchestra.run.smithi139.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.9 MB/s | 43 kB 00:00 2024-09-05T01:50:31.488 INFO:teuthology.orchestra.run.smithi139.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-05T01:50:31.505 INFO:teuthology.orchestra.run.smithi139.stdout:(116/154): python3-certifi-2023.05.07-4.el9.noa 855 kB/s | 14 kB 00:00 2024-09-05T01:50:31.530 INFO:teuthology.orchestra.run.smithi139.stdout:(117/154): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-05T01:50:31.555 INFO:teuthology.orchestra.run.smithi139.stdout:(118/154): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-05T01:50:31.589 INFO:teuthology.orchestra.run.smithi139.stdout:(119/154): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-05T01:50:31.623 INFO:teuthology.orchestra.run.smithi139.stdout:(120/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-05T01:50:31.648 INFO:teuthology.orchestra.run.smithi139.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-05T01:50:31.682 INFO:teuthology.orchestra.run.smithi139.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-05T01:50:31.707 INFO:teuthology.orchestra.run.smithi139.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-05T01:50:31.732 INFO:teuthology.orchestra.run.smithi139.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-05T01:50:31.757 INFO:teuthology.orchestra.run.smithi139.stdout:(125/154): python3-jaraco-collections-3.0.0-8.e 935 kB/s | 23 kB 00:00 2024-09-05T01:50:31.782 INFO:teuthology.orchestra.run.smithi139.stdout:(126/154): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-05T01:50:31.808 INFO:teuthology.orchestra.run.smithi139.stdout:(127/154): python3-jaraco-text-3.2.0-6.el9.noar 782 kB/s | 20 kB 00:00 2024-09-05T01:50:31.867 INFO:teuthology.orchestra.run.smithi139.stdout:(128/154): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-05T01:50:31.892 INFO:teuthology.orchestra.run.smithi139.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-05T01:50:31.934 INFO:teuthology.orchestra.run.smithi139.stdout:(130/154): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-09-05T01:50:31.960 INFO:teuthology.orchestra.run.smithi139.stdout:(131/154): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-09-05T01:50:31.985 INFO:teuthology.orchestra.run.smithi139.stdout:(132/154): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-05T01:50:32.043 INFO:teuthology.orchestra.run.smithi139.stdout:(133/154): libarrow-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-09-05T01:50:32.055 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-09-05T01:50:32.055 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-09-05T01:50:32.068 INFO:teuthology.orchestra.run.smithi139.stdout:(134/154): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-09-05T01:50:32.085 INFO:teuthology.orchestra.run.smithi139.stdout:(135/154): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-05T01:50:32.110 INFO:teuthology.orchestra.run.smithi139.stdout:(136/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-05T01:50:32.127 INFO:teuthology.orchestra.run.smithi139.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 736 kB/s | 31 kB 00:00 2024-09-05T01:50:32.153 INFO:teuthology.orchestra.run.smithi139.stdout:(138/154): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-09-05T01:50:32.178 INFO:teuthology.orchestra.run.smithi139.stdout:(139/154): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-05T01:50:32.203 INFO:teuthology.orchestra.run.smithi139.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-05T01:50:32.228 INFO:teuthology.orchestra.run.smithi139.stdout:(141/154): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-05T01:50:32.254 INFO:teuthology.orchestra.run.smithi139.stdout:(142/154): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-09-05T01:50:32.279 INFO:teuthology.orchestra.run.smithi139.stdout:(143/154): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-05T01:50:32.304 INFO:teuthology.orchestra.run.smithi139.stdout:(144/154): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-05T01:50:32.338 INFO:teuthology.orchestra.run.smithi139.stdout:(145/154): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-05T01:50:32.372 INFO:teuthology.orchestra.run.smithi139.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-05T01:50:32.422 INFO:teuthology.orchestra.run.smithi139.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-09-05T01:50:32.473 INFO:teuthology.orchestra.run.smithi139.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-05T01:50:32.606 INFO:teuthology.orchestra.run.smithi139.stdout:(149/154): NetworkManager-1.48.10-1.el9.x86_64. 17 MB/s | 2.3 MB 00:00 2024-09-05T01:50:32.673 INFO:teuthology.orchestra.run.smithi139.stdout:(150/154): NetworkManager-libnm-1.48.10-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-05T01:50:32.723 INFO:teuthology.orchestra.run.smithi139.stdout:(151/154): librados2-19.1.1-259.g91c8a7e5.el9.x 9.7 MB/s | 3.4 MB 00:00 2024-09-05T01:50:32.748 INFO:teuthology.orchestra.run.smithi139.stdout:(152/154): NetworkManager-team-1.48.10-1.el9.x8 539 kB/s | 40 kB 00:00 2024-09-05T01:50:32.807 INFO:teuthology.orchestra.run.smithi139.stdout:(153/154): librbd1-19.1.1-259.g91c8a7e5.el9.x86 8.2 MB/s | 3.2 MB 00:00 2024-09-05T01:50:32.857 INFO:teuthology.orchestra.run.smithi139.stdout:(154/154): NetworkManager-tui-1.48.10-1.el9.x86 1.8 MB/s | 248 kB 00:00 2024-09-05T01:50:32.863 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:50:32.863 INFO:teuthology.orchestra.run.smithi139.stdout:Total 22 MB/s | 222 MB 00:10 2024-09-05T01:50:34.056 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-05T01:50:34.190 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-05T01:50:34.190 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-05T01:50:34.574 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-05T01:50:35.937 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-05T01:50:35.937 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-05T01:50:36.829 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:50:36.830 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:50:36.844 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T01:50:36.845 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T01:50:36.846 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T01:50:36.847 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T01:50:36.849 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T01:50:36.851 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T01:50:36.852 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T01:50:36.915 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-05T01:50:36.923 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-05T01:50:36.923 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-05T01:50:36.923 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 6.6 k 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.4 M 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 896 k 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 145 k 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 1.5 M 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 146 k 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 3.6 M 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 7.4 M 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 49 k 2024-09-05T01:50:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 11 M 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 49 M 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 285 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 765 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 224 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 27 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 18 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 32 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 752 k 2024-09-05T01:50:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 127 k 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 162 k 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 321 k 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 301 k 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 99 k 2024-09-05T01:50:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 85 k 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.1 M 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 170 k 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.4 M 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.2 M 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T01:50:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 21 M 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 32 k 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 2.4 M 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 249 k 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 4.7 M 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 17 M 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 16 k 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 25 k 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T01:50:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 163 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 506 k 2024-09-05T01:50:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.3 M 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T01:50:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 45 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 136 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T01:50:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T01:50:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T01:50:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T01:50:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T01:50:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout:Install 148 Packages 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 6 Packages 2024-09-05T01:50:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:50:36.937 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 222 M 2024-09-05T01:50:36.937 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-05T01:50:37.918 INFO:teuthology.orchestra.run.smithi103.stdout:(1/154): ceph-19.1.1-259.g91c8a7e5.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-05T01:50:38.135 INFO:teuthology.orchestra.run.smithi103.stdout:(2/154): ceph-fuse-19.1.1-259.g91c8a7e5.el9.x86 4.0 MB/s | 896 kB 00:00 2024-09-05T01:50:38.235 INFO:teuthology.orchestra.run.smithi103.stdout:(3/154): ceph-base-19.1.1-259.g91c8a7e5.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-05T01:50:38.260 INFO:teuthology.orchestra.run.smithi103.stdout:(4/154): ceph-immutable-object-cache-19.1.1-259 1.1 MB/s | 145 kB 00:00 2024-09-05T01:50:38.272 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-09-05T01:50:38.378 INFO:teuthology.orchestra.run.smithi103.stdout:(5/154): ceph-mds-19.1.1-259.g91c8a7e5.el9.x86_ 17 MB/s | 2.4 MB 00:00 2024-09-05T01:50:38.397 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T01:50:38.502 INFO:teuthology.orchestra.run.smithi103.stdout:(6/154): ceph-mgr-19.1.1-259.g91c8a7e5.el9.x86_ 6.3 MB/s | 1.5 MB 00:00 2024-09-05T01:50:38.653 INFO:teuthology.orchestra.run.smithi103.stdout:(7/154): ceph-mon-19.1.1-259.g91c8a7e5.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-05T01:50:38.679 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T01:50:38.723 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T01:50:38.986 INFO:teuthology.orchestra.run.smithi103.stdout:(8/154): ceph-common-19.1.1-259.g91c8a7e5.el9.x 18 MB/s | 21 MB 00:01 2024-09-05T01:50:39.028 INFO:teuthology.orchestra.run.smithi103.stdout:(9/154): ceph-selinux-19.1.1-259.g91c8a7e5.el9. 607 kB/s | 25 kB 00:00 2024-09-05T01:50:39.055 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T01:50:39.171 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:50:39.229 INFO:teuthology.orchestra.run.smithi103.stdout:(10/154): ceph-radosgw-19.1.1-259.g91c8a7e5.el9 18 MB/s | 11 MB 00:00 2024-09-05T01:50:39.276 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:50:39.295 INFO:teuthology.orchestra.run.smithi103.stdout:(11/154): cephfs-mirror-19.1.1-259.g91c8a7e5.el 3.3 MB/s | 224 kB 00:00 2024-09-05T01:50:39.306 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:50:39.388 INFO:teuthology.orchestra.run.smithi103.stdout:(12/154): cephfs-shell-19.1.1-259.g91c8a7e5.el9 295 kB/s | 27 kB 00:00 2024-09-05T01:50:39.398 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:50:39.440 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:50:39.469 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T01:50:39.488 INFO:teuthology.orchestra.run.smithi103.stdout:(13/154): libcephfs-devel-19.1.1-259.g91c8a7e5. 318 kB/s | 32 kB 00:00 2024-09-05T01:50:39.495 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T01:50:39.597 INFO:teuthology.orchestra.run.smithi103.stdout:(14/154): libcephfs2-19.1.1-259.g91c8a7e5.el9.x 6.8 MB/s | 752 kB 00:00 2024-09-05T01:50:39.653 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T01:50:39.697 INFO:teuthology.orchestra.run.smithi103.stdout:(15/154): libcephsqlite-19.1.1-259.g91c8a7e5.el 1.6 MB/s | 163 kB 00:00 2024-09-05T01:50:39.700 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T01:50:39.737 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:50:39.833 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:50:39.863 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:50:39.872 INFO:teuthology.orchestra.run.smithi103.stdout:(16/154): ceph-osd-19.1.1-259.g91c8a7e5.el9.x86 12 MB/s | 17 MB 00:01 2024-09-05T01:50:39.898 INFO:teuthology.orchestra.run.smithi103.stdout:(17/154): librados-devel-19.1.1-259.g91c8a7e5.e 631 kB/s | 127 kB 00:00 2024-09-05T01:50:39.909 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:50:39.931 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:50:39.931 INFO:teuthology.orchestra.run.smithi103.stdout:(18/154): libradosstriper1-19.1.1-259.g91c8a7e5 8.4 MB/s | 506 kB 00:00 2024-09-05T01:50:39.957 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:50:40.023 INFO:teuthology.orchestra.run.smithi103.stdout:(19/154): python3-ceph-argparse-19.1.1-259.g91c 492 kB/s | 45 kB 00:00 2024-09-05T01:50:40.132 INFO:teuthology.orchestra.run.smithi103.stdout:(20/154): python3-ceph-common-19.1.1-259.g91c8a 1.2 MB/s | 136 kB 00:00 2024-09-05T01:50:40.158 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:50:40.233 INFO:teuthology.orchestra.run.smithi103.stdout:(21/154): librgw2-19.1.1-259.g91c8a7e5.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-05T01:50:40.266 INFO:teuthology.orchestra.run.smithi103.stdout:(22/154): python3-cephfs-19.1.1-259.g91c8a7e5.e 1.2 MB/s | 162 kB 00:00 2024-09-05T01:50:40.291 INFO:teuthology.orchestra.run.smithi103.stdout:(23/154): python3-rados-19.1.1-259.g91c8a7e5.el 5.4 MB/s | 321 kB 00:00 2024-09-05T01:50:40.317 INFO:teuthology.orchestra.run.smithi103.stdout:(24/154): python3-rbd-19.1.1-259.g91c8a7e5.el9. 5.9 MB/s | 301 kB 00:00 2024-09-05T01:50:40.319 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:50:40.350 INFO:teuthology.orchestra.run.smithi103.stdout:(25/154): python3-rgw-19.1.1-259.g91c8a7e5.el9. 1.7 MB/s | 99 kB 00:00 2024-09-05T01:50:40.375 INFO:teuthology.orchestra.run.smithi103.stdout:(26/154): rbd-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-05T01:50:40.434 INFO:teuthology.orchestra.run.smithi103.stdout:(27/154): rbd-nbd-19.1.1-259.g91c8a7e5.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-05T01:50:40.445 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:50:40.506 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:50:40.518 INFO:teuthology.orchestra.run.smithi103.stdout:(28/154): ceph-grafana-dashboards-19.1.1-259.g9 383 kB/s | 32 kB 00:00 2024-09-05T01:50:40.585 INFO:teuthology.orchestra.run.smithi103.stdout:(29/154): rbd-mirror-19.1.1-259.g91c8a7e5.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-05T01:50:40.610 INFO:teuthology.orchestra.run.smithi103.stdout:(30/154): ceph-mgr-cephadm-19.1.1-259.g91c8a7e5 1.6 MB/s | 146 kB 00:00 2024-09-05T01:50:40.618 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:50:40.724 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:50:40.919 INFO:teuthology.orchestra.run.smithi103.stdout:(31/154): ceph-mgr-dashboard-19.1.1-259.g91c8a7 11 MB/s | 3.6 MB 00:00 2024-09-05T01:50:40.954 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:50:41.003 INFO:teuthology.orchestra.run.smithi103.stdout:(32/154): ceph-mgr-modules-core-19.1.1-259.g91c 2.9 MB/s | 249 kB 00:00 2024-09-05T01:50:41.075 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T01:50:41.153 INFO:teuthology.orchestra.run.smithi103.stdout:(33/154): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-05T01:50:41.170 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T01:50:41.178 INFO:teuthology.orchestra.run.smithi103.stdout:(34/154): ceph-mgr-rook-19.1.1-259.g91c8a7e5.el 281 kB/s | 49 kB 00:00 2024-09-05T01:50:41.204 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T01:50:41.219 INFO:teuthology.orchestra.run.smithi103.stdout:(35/154): ceph-prometheus-alerts-19.1.1-259.g91 248 kB/s | 16 kB 00:00 2024-09-05T01:50:41.239 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T01:50:41.254 INFO:teuthology.orchestra.run.smithi103.stdout:(36/154): ceph-volume-19.1.1-259.g91c8a7e5.el9. 3.7 MB/s | 285 kB 00:00 2024-09-05T01:50:41.287 INFO:teuthology.orchestra.run.smithi103.stdout:(37/154): cephadm-19.1.1-259.g91c8a7e5.el9.noar 11 MB/s | 765 kB 00:00 2024-09-05T01:50:41.313 INFO:teuthology.orchestra.run.smithi103.stdout:(38/154): cephfs-top-19.1.1-259.g91c8a7e5.el9.n 311 kB/s | 18 kB 00:00 2024-09-05T01:50:41.314 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T01:50:41.366 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T01:50:41.414 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T01:50:41.466 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T01:50:41.499 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T01:50:41.526 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T01:50:41.597 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T01:50:41.650 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T01:50:41.679 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T01:50:41.715 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T01:50:41.764 INFO:teuthology.orchestra.run.smithi103.stdout:(39/154): ceph-test-19.1.1-259.g91c8a7e5.el9.x8 18 MB/s | 49 MB 00:02 2024-09-05T01:50:41.792 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T01:50:41.826 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 33/160 2024-09-05T01:50:41.872 INFO:teuthology.orchestra.run.smithi103.stdout:(40/154): fuse-2.9.9-16.el9.x86_64.rpm 137 kB/s | 80 kB 00:00 2024-09-05T01:50:41.877 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:50:41.915 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 35/160 2024-09-05T01:50:41.922 INFO:teuthology.orchestra.run.smithi103.stdout:(41/154): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-05T01:50:41.973 INFO:teuthology.orchestra.run.smithi103.stdout:(42/154): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-05T01:50:41.990 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T01:50:41.998 INFO:teuthology.orchestra.run.smithi103.stdout:(43/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-05T01:50:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-05T01:50:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:(44/154): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-05T01:50:42.033 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T01:50:42.065 INFO:teuthology.orchestra.run.smithi103.stdout:(45/154): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-09-05T01:50:42.099 INFO:teuthology.orchestra.run.smithi103.stdout:(46/154): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-05T01:50:42.110 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T01:50:42.124 INFO:teuthology.orchestra.run.smithi103.stdout:(47/154): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-09-05T01:50:42.153 INFO:teuthology.orchestra.run.smithi103.stdout:(48/154): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-05T01:50:42.159 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T01:50:42.256 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T01:50:42.258 INFO:teuthology.orchestra.run.smithi103.stdout:(49/154): boost-program-options-1.75.0-8.el9.x8 988 kB/s | 107 kB 00:00 2024-09-05T01:50:42.293 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T01:50:42.309 INFO:teuthology.orchestra.run.smithi103.stdout:(50/154): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-05T01:50:42.321 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T01:50:42.334 INFO:teuthology.orchestra.run.smithi103.stdout:(51/154): elfutils-libelf-devel-0.191-4.el9.x86 289 kB/s | 22 kB 00:00 2024-09-05T01:50:42.337 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T01:50:42.359 INFO:teuthology.orchestra.run.smithi103.stdout:(52/154): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-05T01:50:42.384 INFO:teuthology.orchestra.run.smithi103.stdout:(53/154): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-05T01:50:42.410 INFO:teuthology.orchestra.run.smithi103.stdout:(54/154): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-05T01:50:42.435 INFO:teuthology.orchestra.run.smithi103.stdout:(55/154): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-05T01:50:42.466 INFO:teuthology.orchestra.run.smithi103.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 11 kB 00:00 2024-09-05T01:50:42.508 INFO:teuthology.orchestra.run.smithi103.stdout:(57/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.2 MB/s | 160 kB 00:00 2024-09-05T01:50:42.533 INFO:teuthology.orchestra.run.smithi103.stdout:(58/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-09-05T01:50:42.567 INFO:teuthology.orchestra.run.smithi103.stdout:(59/154): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-05T01:50:42.609 INFO:teuthology.orchestra.run.smithi103.stdout:(60/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 587 kB/s | 24 kB 00:00 2024-09-05T01:50:42.650 INFO:teuthology.orchestra.run.smithi103.stdout:(61/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-05T01:50:42.673 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T01:50:42.684 INFO:teuthology.orchestra.run.smithi103.stdout:(62/154): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-05T01:50:42.710 INFO:teuthology.orchestra.run.smithi103.stdout:(63/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-05T01:50:42.735 INFO:teuthology.orchestra.run.smithi103.stdout:(64/154): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-05T01:50:42.760 INFO:teuthology.orchestra.run.smithi103.stdout:(65/154): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-05T01:50:42.802 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:50:42.876 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T01:50:42.902 INFO:teuthology.orchestra.run.smithi103.stdout:(66/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-09-05T01:50:42.933 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:50:42.967 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T01:50:42.973 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:50:42.978 INFO:teuthology.orchestra.run.smithi103.stdout:(67/154): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-05T01:50:43.002 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T01:50:43.028 INFO:teuthology.orchestra.run.smithi103.stdout:(68/154): postgresql-contrib-13.16-1.el9.x86_64 18 MB/s | 910 kB 00:00 2024-09-05T01:50:43.045 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T01:50:43.062 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:50:43.070 INFO:teuthology.orchestra.run.smithi103.stdout:(69/154): postgresql-private-libs-13.16-1.el9.x 3.2 MB/s | 137 kB 00:00 2024-09-05T01:50:43.084 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 45/160 2024-09-05T01:50:43.103 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:50:43.117 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T01:50:43.132 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T01:50:43.153 INFO:teuthology.orchestra.run.smithi103.stdout:(70/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-09-05T01:50:43.158 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T01:50:43.173 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T01:50:43.247 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T01:50:43.254 INFO:teuthology.orchestra.run.smithi103.stdout:(71/154): postgresql-server-13.16-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-09-05T01:50:43.285 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T01:50:43.296 INFO:teuthology.orchestra.run.smithi103.stdout:(72/154): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-09-05T01:50:43.312 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T01:50:43.321 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T01:50:43.355 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T01:50:43.371 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T01:50:43.391 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:50:43.413 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T01:50:43.446 INFO:teuthology.orchestra.run.smithi103.stdout:(73/154): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-09-05T01:50:43.456 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T01:50:43.480 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:50:43.488 INFO:teuthology.orchestra.run.smithi103.stdout:(74/154): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-05T01:50:43.498 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T01:50:43.510 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:50:43.538 INFO:teuthology.orchestra.run.smithi103.stdout:(75/154): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-05T01:50:43.572 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:50:43.580 INFO:teuthology.orchestra.run.smithi103.stdout:(76/154): python3-jinja2-2.11.3-6.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-09-05T01:50:43.602 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:50:43.626 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:50:43.631 INFO:teuthology.orchestra.run.smithi103.stdout:(77/154): protobuf-3.14.0-13.el9.x86_64.rpm 2.1 MB/s | 1.0 MB 00:00 2024-09-05T01:50:43.648 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T01:50:43.656 INFO:teuthology.orchestra.run.smithi103.stdout:(78/154): python3-jsonpatch-1.21-16.el9.noarch. 339 kB/s | 26 kB 00:00 2024-09-05T01:50:43.681 INFO:teuthology.orchestra.run.smithi103.stdout:(79/154): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-05T01:50:43.706 INFO:teuthology.orchestra.run.smithi103.stdout:(80/154): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-05T01:50:43.732 INFO:teuthology.orchestra.run.smithi103.stdout:(81/154): python3-markupsafe-1.1.1-12.el9.x86_6 689 kB/s | 35 kB 00:00 2024-09-05T01:50:43.849 INFO:teuthology.orchestra.run.smithi103.stdout:(82/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-05T01:50:43.875 INFO:teuthology.orchestra.run.smithi103.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-09-05T01:50:43.878 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T01:50:43.900 INFO:teuthology.orchestra.run.smithi103.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-05T01:50:43.920 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T01:50:43.942 INFO:teuthology.orchestra.run.smithi103.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-05T01:50:43.959 INFO:teuthology.orchestra.run.smithi103.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-09-05T01:50:44.001 INFO:teuthology.orchestra.run.smithi103.stdout:(87/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-09-05T01:50:44.028 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T01:50:44.038 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:50:44.043 INFO:teuthology.orchestra.run.smithi103.stdout:(88/154): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-09-05T01:50:44.060 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T01:50:44.068 INFO:teuthology.orchestra.run.smithi103.stdout:(89/154): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-09-05T01:50:44.093 INFO:teuthology.orchestra.run.smithi103.stdout:(90/154): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-05T01:50:44.106 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T01:50:44.119 INFO:teuthology.orchestra.run.smithi103.stdout:(91/154): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-05T01:50:44.131 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T01:50:44.161 INFO:teuthology.orchestra.run.smithi103.stdout:(92/154): python3-toml-0.10.2-6.el9.noarch.rpm 996 kB/s | 42 kB 00:00 2024-09-05T01:50:44.181 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T01:50:44.188 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:50:44.203 INFO:teuthology.orchestra.run.smithi103.stdout:(93/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-05T01:50:44.258 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T01:50:44.303 INFO:teuthology.orchestra.run.smithi103.stdout:(94/154): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-09-05T01:50:44.316 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:50:44.353 INFO:teuthology.orchestra.run.smithi103.stdout:(95/154): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-05T01:50:44.376 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:50:44.402 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T01:50:44.447 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T01:50:44.475 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T01:50:44.492 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:50:44.509 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T01:50:44.520 INFO:teuthology.orchestra.run.smithi103.stdout:(96/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-09-05T01:50:44.546 INFO:teuthology.orchestra.run.smithi103.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-09-05T01:50:44.554 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T01:50:44.569 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:50:44.592 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T01:50:44.613 INFO:teuthology.orchestra.run.smithi103.stdout:(98/154): openblas-openmp-0.3.26-2.el9.x86_64.r 2.6 MB/s | 4.9 MB 00:01 2024-09-05T01:50:44.631 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T01:50:44.638 INFO:teuthology.orchestra.run.smithi103.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 629 kB/s | 58 kB 00:00 2024-09-05T01:50:44.661 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T01:50:44.672 INFO:teuthology.orchestra.run.smithi103.stdout:(100/154): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-05T01:50:44.696 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T01:50:44.697 INFO:teuthology.orchestra.run.smithi103.stdout:(101/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-05T01:50:44.722 INFO:teuthology.orchestra.run.smithi103.stdout:(102/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 759 kB/s | 64 kB 00:00 2024-09-05T01:50:44.729 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T01:50:44.747 INFO:teuthology.orchestra.run.smithi103.stdout:(103/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-05T01:50:44.771 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T01:50:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:50:44.805 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T01:50:44.823 INFO:teuthology.orchestra.run.smithi103.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-05T01:50:44.848 INFO:teuthology.orchestra.run.smithi103.stdout:(105/154): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-05T01:50:44.873 INFO:teuthology.orchestra.run.smithi103.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 772 kB/s | 19 kB 00:00 2024-09-05T01:50:44.890 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T01:50:44.895 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T01:50:44.945 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T01:50:44.974 INFO:teuthology.orchestra.run.smithi103.stdout:(107/154): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-05T01:50:44.976 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T01:50:44.987 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T01:50:45.015 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T01:50:45.016 INFO:teuthology.orchestra.run.smithi103.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 619 kB/s | 26 kB 00:00 2024-09-05T01:50:45.041 INFO:teuthology.orchestra.run.smithi103.stdout:(109/154): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-09-05T01:50:45.058 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T01:50:45.075 INFO:teuthology.orchestra.run.smithi103.stdout:(110/154): liboath-2.6.7-2.el9.x86_64.rpm 827 kB/s | 49 kB 00:00 2024-09-05T01:50:45.089 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T01:50:45.100 INFO:teuthology.orchestra.run.smithi103.stdout:(111/154): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-05T01:50:45.126 INFO:teuthology.orchestra.run.smithi103.stdout:(112/154): libunwind-1.6.2-1.el9.x86_64.rpm 801 kB/s | 67 kB 00:00 2024-09-05T01:50:45.140 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T01:50:45.161 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T01:50:45.168 INFO:teuthology.orchestra.run.smithi103.stdout:(113/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-05T01:50:45.184 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T01:50:45.193 INFO:teuthology.orchestra.run.smithi103.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-05T01:50:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T01:50:45.218 INFO:teuthology.orchestra.run.smithi103.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-05T01:50:45.243 INFO:teuthology.orchestra.run.smithi103.stdout:(116/154): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-05T01:50:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T01:50:45.269 INFO:teuthology.orchestra.run.smithi103.stdout:(117/154): python3-asyncssh-2.13.2-5.el9.noarch 3.7 MB/s | 548 kB 00:00 2024-09-05T01:50:45.294 INFO:teuthology.orchestra.run.smithi103.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-05T01:50:45.300 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T01:50:45.328 INFO:teuthology.orchestra.run.smithi103.stdout:(119/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-05T01:50:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T01:50:45.353 INFO:teuthology.orchestra.run.smithi103.stdout:(120/154): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-05T01:50:45.379 INFO:teuthology.orchestra.run.smithi103.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-05T01:50:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T01:50:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-05T01:50:45.429 INFO:teuthology.orchestra.run.smithi103.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-05T01:50:45.451 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T01:50:45.454 INFO:teuthology.orchestra.run.smithi103.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 704 kB/s | 18 kB 00:00 2024-09-05T01:50:45.479 INFO:teuthology.orchestra.run.smithi103.stdout:(125/154): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-09-05T01:50:45.489 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T01:50:45.513 INFO:teuthology.orchestra.run.smithi103.stdout:(126/154): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-09-05T01:50:45.524 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T01:50:45.539 INFO:teuthology.orchestra.run.smithi103.stdout:(127/154): python3-jaraco-functools-3.5.0-2.el9 329 kB/s | 19 kB 00:00 2024-09-05T01:50:45.601 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T01:50:45.605 INFO:teuthology.orchestra.run.smithi103.stdout:(128/154): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-09-05T01:50:45.631 INFO:teuthology.orchestra.run.smithi103.stdout:(129/154): python3-jaraco-text-3.2.0-6.el9.noar 167 kB/s | 20 kB 00:00 2024-09-05T01:50:45.635 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 33/160 2024-09-05T01:50:45.656 INFO:teuthology.orchestra.run.smithi103.stdout:(130/154): python3-logutils-0.3.5-21.el9.noarch 922 kB/s | 46 kB 00:00 2024-09-05T01:50:45.682 INFO:teuthology.orchestra.run.smithi103.stdout:(131/154): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-05T01:50:45.682 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:50:45.707 INFO:teuthology.orchestra.run.smithi103.stdout:(132/154): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-05T01:50:45.727 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 35/160 2024-09-05T01:50:45.732 INFO:teuthology.orchestra.run.smithi103.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-05T01:50:45.766 INFO:teuthology.orchestra.run.smithi103.stdout:(134/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-09-05T01:50:45.798 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T01:50:45.800 INFO:teuthology.orchestra.run.smithi103.stdout:(135/154): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-05T01:50:45.825 INFO:teuthology.orchestra.run.smithi103.stdout:(136/154): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-09-05T01:50:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T01:50:45.850 INFO:teuthology.orchestra.run.smithi103.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-05T01:50:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:(138/154): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-09-05T01:50:45.901 INFO:teuthology.orchestra.run.smithi103.stdout:(139/154): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-05T01:50:45.913 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T01:50:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T01:50:45.918 INFO:teuthology.orchestra.run.smithi103.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-05T01:50:45.943 INFO:teuthology.orchestra.run.smithi103.stdout:(141/154): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-05T01:50:45.969 INFO:teuthology.orchestra.run.smithi103.stdout:(142/154): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-05T01:50:45.974 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T01:50:45.994 INFO:teuthology.orchestra.run.smithi103.stdout:(143/154): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-05T01:50:46.008 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T01:50:46.019 INFO:teuthology.orchestra.run.smithi103.stdout:(144/154): python3-zc-lockfile-2.0-10.el9.noarc 799 kB/s | 20 kB 00:00 2024-09-05T01:50:46.045 INFO:teuthology.orchestra.run.smithi103.stdout:(145/154): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-05T01:50:46.051 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T01:50:46.070 INFO:teuthology.orchestra.run.smithi103.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-05T01:50:46.104 INFO:teuthology.orchestra.run.smithi103.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-05T01:50:46.104 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T01:50:46.111 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T01:50:46.154 INFO:teuthology.orchestra.run.smithi103.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-05T01:50:46.172 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T01:50:46.217 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T01:50:46.304 INFO:teuthology.orchestra.run.smithi103.stdout:(149/154): NetworkManager-1.48.10-1.el9.x86_64. 16 MB/s | 2.3 MB 00:00 2024-09-05T01:50:46.306 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T01:50:46.344 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T01:50:46.363 INFO:teuthology.orchestra.run.smithi103.stdout:(150/154): librados2-19.1.1-259.g91c8a7e5.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-05T01:50:46.405 INFO:teuthology.orchestra.run.smithi103.stdout:(151/154): NetworkManager-libnm-1.48.10-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-09-05T01:50:46.430 INFO:teuthology.orchestra.run.smithi103.stdout:(152/154): NetworkManager-team-1.48.10-1.el9.x8 607 kB/s | 40 kB 00:00 2024-09-05T01:50:46.455 INFO:teuthology.orchestra.run.smithi103.stdout:(153/154): NetworkManager-tui-1.48.10-1.el9.x86 4.8 MB/s | 248 kB 00:00 2024-09-05T01:50:46.530 INFO:teuthology.orchestra.run.smithi103.stdout:(154/154): librbd1-19.1.1-259.g91c8a7e5.el9.x86 7.4 MB/s | 3.2 MB 00:00 2024-09-05T01:50:46.532 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T01:50:46.537 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:50:46.537 INFO:teuthology.orchestra.run.smithi103.stdout:Total 23 MB/s | 222 MB 00:09 2024-09-05T01:50:46.558 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T01:50:46.637 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T01:50:46.651 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T01:50:46.733 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T01:50:46.761 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T01:50:46.804 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T01:50:46.842 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 45/160 2024-09-05T01:50:46.883 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T01:50:46.947 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T01:50:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T01:50:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T01:50:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T01:50:47.134 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T01:50:47.170 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T01:50:47.213 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T01:50:47.313 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T01:50:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T01:50:47.436 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T01:50:47.503 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T01:50:47.549 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T01:50:47.590 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T01:50:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T01:50:47.705 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-05T01:50:47.748 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T01:50:47.788 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T01:50:47.813 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T01:50:47.837 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T01:50:47.839 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-05T01:50:47.839 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-05T01:50:47.912 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T01:50:48.098 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T01:50:48.136 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T01:50:48.164 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T01:50:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T01:50:48.243 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T01:50:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T01:50:48.312 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T01:50:48.350 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T01:50:48.393 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T01:50:48.427 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T01:50:48.468 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T01:50:48.502 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T01:50:48.590 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T01:50:48.641 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T01:50:48.672 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T01:50:48.708 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T01:50:48.777 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T01:50:48.798 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T01:50:48.826 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T01:50:48.832 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T01:50:48.859 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T01:50:49.171 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T01:50:49.255 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:50:49.546 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:50:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T01:50:49.602 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-05T01:50:49.603 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-05T01:50:49.619 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T01:50:49.653 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T01:50:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 100/160 2024-09-05T01:50:49.721 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T01:50:49.762 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T01:50:49.834 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T01:50:49.887 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T01:50:49.975 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T01:50:50.006 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T01:50:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T01:50:50.204 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T01:50:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T01:50:50.233 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:50:50.289 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T01:50:50.305 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:50:50.352 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 104/160 2024-09-05T01:50:50.952 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T01:50:51.077 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T01:50:52.336 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T01:50:52.584 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T01:50:52.650 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:50:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T01:50:52.682 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:50:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T01:50:53.012 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T01:50:53.058 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:50:53.130 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:50:53.162 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 100/160 2024-09-05T01:50:53.198 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T01:50:53.228 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:50:53.269 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:50:53.364 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:50:53.364 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T01:50:53.365 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T01:50:53.365 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:50:53.396 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:50:53.677 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T01:50:53.702 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:50:53.773 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:50:53.804 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 104/160 2024-09-05T01:50:55.732 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-05T01:50:55.816 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T01:50:55.943 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T01:50:56.116 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T01:50:56.130 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:50:56.167 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:50:56.454 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T01:50:56.527 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:50:56.649 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:50:56.686 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:50:56.708 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:50:56.754 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:50:56.762 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:50:56.803 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:50:56.833 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T01:50:56.848 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:50:56.848 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T01:50:56.848 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T01:50:56.848 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:50:56.858 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T01:50:56.880 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:50:57.022 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T01:50:57.072 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T01:50:57.109 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:50:57.196 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:50:57.218 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:50:57.273 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:50:57.294 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:50:57.313 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:50:57.505 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:50:57.656 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:50:57.792 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:50:57.852 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:50:57.953 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:50:58.054 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:50:58.281 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:50:58.391 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T01:50:58.489 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T01:50:58.525 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T01:50:58.561 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T01:50:58.634 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T01:50:58.687 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T01:50:58.735 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T01:50:58.778 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T01:50:58.811 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T01:50:58.839 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T01:50:58.909 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T01:50:58.963 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T01:50:58.992 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T01:50:59.027 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T01:50:59.130 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T01:50:59.172 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 33/160 2024-09-05T01:50:59.210 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:50:59.240 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 35/160 2024-09-05T01:50:59.312 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T01:50:59.364 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T01:50:59.447 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T01:50:59.593 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T01:50:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T01:51:00.192 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T01:51:00.280 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T01:51:00.316 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T01:51:00.358 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T01:51:00.397 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 45/160 2024-09-05T01:51:00.430 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T01:51:00.486 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T01:51:00.560 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T01:51:00.598 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T01:51:00.643 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T01:51:00.699 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T01:51:00.734 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T01:51:00.777 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T01:51:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T01:51:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T01:51:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T01:51:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T01:51:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T01:51:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T01:51:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T01:51:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T01:51:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T01:51:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T01:51:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T01:51:01.552 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T01:51:01.588 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T01:51:01.631 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T01:51:01.675 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T01:51:01.713 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T01:51:01.744 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T01:51:01.774 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T01:51:01.809 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T01:51:01.843 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T01:51:01.884 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T01:51:01.919 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T01:51:02.007 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T01:51:02.058 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T01:51:02.089 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T01:51:02.193 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T01:51:02.248 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T01:51:02.989 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T01:51:03.048 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T01:51:03.079 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T01:51:03.184 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T01:51:03.235 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T01:51:03.280 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T01:51:03.369 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T01:51:03.407 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T01:51:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T01:51:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T01:51:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T01:51:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T01:51:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T01:51:05.247 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:51:05.248 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-09-05T01:51:05.248 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-09-05T01:51:05.248 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-09-05T01:51:05.248 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-09-05T01:51:05.248 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-09-05T01:51:05.248 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-09-05T01:51:05.248 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-09-05T01:51:05.248 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:05.470 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:51:05.486 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:51:05.486 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:05.487 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T01:51:05.487 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:05.753 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T01:51:05.853 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T01:51:05.881 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T01:51:05.914 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T01:51:06.223 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T01:51:06.285 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:51:06.368 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:51:06.398 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 100/160 2024-09-05T01:51:06.434 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T01:51:06.922 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T01:51:06.955 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:51:06.982 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:51:07.008 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:51:07.009 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:07.009 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T01:51:07.009 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:51:07.009 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:51:07.009 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:07.026 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:51:07.057 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 104/160 2024-09-05T01:51:07.240 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:51:07.264 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:51:07.264 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:07.264 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T01:51:07.264 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:51:07.264 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:51:07.264 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:51:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:51:07.723 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:07.723 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T01:51:07.723 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:51:07.724 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:51:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:07.838 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T01:51:07.904 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T01:51:07.960 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T01:51:08.148 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T01:51:08.201 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T01:51:08.288 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T01:51:08.385 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T01:51:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:51:08.414 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-09-05T01:51:08.414 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-09-05T01:51:08.414 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-09-05T01:51:08.414 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-09-05T01:51:08.414 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-09-05T01:51:08.414 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-09-05T01:51:08.414 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-09-05T01:51:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T01:51:08.568 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T01:51:08.597 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:51:08.608 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T01:51:08.614 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:51:08.614 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:08.614 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T01:51:08.614 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:08.636 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T01:51:09.057 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:51:09.085 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:51:09.116 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 124/160 2024-09-05T01:51:09.180 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 125/160 2024-09-05T01:51:09.217 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:51:09.279 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:51:09.321 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:51:09.868 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:51:09.907 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:51:09.999 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:51:09.999 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T01:51:09.999 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T01:51:09.999 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:10.025 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:51:10.058 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:51:10.085 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:51:10.085 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:10.086 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T01:51:10.086 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:51:10.086 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:51:10.086 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:10.269 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:51:10.309 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:51:10.310 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:51:10.332 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:51:10.333 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:10.333 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T01:51:10.333 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:51:10.333 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:51:10.333 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:10.768 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:51:10.791 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:51:10.792 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:10.792 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T01:51:10.792 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:51:10.792 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:51:10.792 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T01:51:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T01:51:11.027 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T01:51:11.212 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T01:51:11.277 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T01:51:11.277 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:51:11.302 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:51:11.362 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T01:51:11.407 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:51:11.459 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T01:51:11.573 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 129/160 2024-09-05T01:51:11.598 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T01:51:11.605 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:51:11.629 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:51:11.629 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:11.629 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T01:51:11.629 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:51:11.630 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:51:11.630 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:11.635 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T01:51:11.667 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T01:51:11.680 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:51:11.693 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:51:11.695 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T01:51:11.791 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 132/160 2024-09-05T01:51:11.837 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 133/160 2024-09-05T01:51:12.157 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:51:12.193 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:51:12.233 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 124/160 2024-09-05T01:51:12.295 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 125/160 2024-09-05T01:51:12.324 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:51:12.776 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T01:51:12.825 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:51:12.849 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:51:12.849 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:12.849 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T01:51:12.849 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:51:12.850 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:51:12.850 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:12.892 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:51:12.915 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:51:12.916 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:12.916 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T01:51:12.916 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:12.968 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:51:12.992 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:51:12.992 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:12.992 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T01:51:12.992 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:51:12.992 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:51:12.992 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:13.302 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:51:13.326 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:51:13.326 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:13.326 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T01:51:13.326 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:51:13.326 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:51:13.327 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:13.366 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:51:13.401 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:51:14.365 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:51:14.402 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:51:14.503 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:51:14.663 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 129/160 2024-09-05T01:51:14.687 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:51:14.711 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:51:14.711 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:14.711 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T01:51:14.711 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:51:14.711 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:51:14.711 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:14.763 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:51:14.774 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:51:14.872 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 132/160 2024-09-05T01:51:14.936 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 133/160 2024-09-05T01:51:15.862 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T01:51:15.890 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:51:15.914 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:51:15.914 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:15.914 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T01:51:15.914 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:51:15.914 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:51:15.914 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:15.958 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:51:15.980 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:51:15.980 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:15.980 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T01:51:15.980 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:51:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:51:16.048 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:16.048 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T01:51:16.048 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:51:16.048 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:51:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:16.350 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:51:16.374 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:51:16.374 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:16.374 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T01:51:16.374 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:51:16.374 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:51:16.374 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:17.881 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 139/160 2024-09-05T01:51:17.923 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 140/160 2024-09-05T01:51:17.964 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 141/160 2024-09-05T01:51:17.996 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T01:51:18.032 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T01:51:18.222 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 144/160 2024-09-05T01:51:18.270 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T01:51:18.309 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 146/160 2024-09-05T01:51:18.347 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T01:51:18.393 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T01:51:19.087 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:51:19.126 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:51:19.146 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:51:19.993 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 150/160 2024-09-05T01:51:20.081 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T01:51:20.122 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T01:51:20.206 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T01:51:20.235 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T01:51:20.261 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T01:51:20.261 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:51:20.362 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:51:20.395 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T01:51:20.401 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:20.401 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:20.580 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:20.581 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:51:20.673 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:51:20.673 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:51:20.912 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 139/160 2024-09-05T01:51:20.953 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 140/160 2024-09-05T01:51:20.994 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 141/160 2024-09-05T01:51:21.026 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T01:51:21.062 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T01:51:21.250 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 144/160 2024-09-05T01:51:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T01:51:21.322 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 146/160 2024-09-05T01:51:21.352 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T01:51:21.398 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T01:51:21.793 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:51:21.794 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-09-05T01:51:21.794 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-09-05T01:51:21.794 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-09-05T01:51:21.794 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-09-05T01:51:21.794 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-09-05T01:51:21.794 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-09-05T01:51:21.794 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-09-05T01:51:21.794 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:21.965 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:51:21.981 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:51:21.981 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:21.981 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T01:51:21.981 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:21.986 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:51:22.029 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:51:22.049 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:51:22.878 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 150/160 2024-09-05T01:51:22.966 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T01:51:23.007 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T01:51:23.097 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T01:51:23.128 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T01:51:23.154 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T01:51:23.154 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:51:23.238 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:51:23.262 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T01:51:23.268 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:23.268 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:23.448 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:23.448 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:51:23.451 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:51:23.478 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:51:23.478 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:23.478 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T01:51:23.478 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:51:23.478 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:51:23.479 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:23.532 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:51:23.532 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:51:23.703 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:51:23.727 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:51:23.727 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:23.727 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T01:51:23.727 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:51:23.727 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:51:23.727 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:24.171 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:51:24.195 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:51:24.195 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:24.195 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T01:51:24.195 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:51:24.195 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:51:24.196 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:24.310 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T01:51:24.366 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T01:51:24.422 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T01:51:24.607 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T01:51:24.664 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T01:51:24.742 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T01:51:24.833 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T01:51:24.978 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T01:51:25.015 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T01:51:25.055 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T01:51:25.091 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T01:51:25.553 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:51:25.582 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:51:25.613 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 124/160 2024-09-05T01:51:25.676 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 125/160 2024-09-05T01:51:25.705 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:51:26.745 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:51:26.796 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:51:27.074 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:51:27.074 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 1/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 3/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 4/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 5/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 8/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 9/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 10/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 11/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 13/160 2024-09-05T01:51:27.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 14/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 15/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 16/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 17/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 18/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 19/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 20/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 21/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 22/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 23/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 24/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 25/160 2024-09-05T01:51:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 26/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 27/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 28/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 29/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 30/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 31/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 32/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 33/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 35/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 36/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 37/160 2024-09-05T01:51:27.077 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 38/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 39/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T01:51:27.078 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T01:51:27.079 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T01:51:27.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T01:51:27.081 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T01:51:27.082 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T01:51:27.083 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T01:51:27.084 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T01:51:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 149/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T01:51:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 151/160 2024-09-05T01:51:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T01:51:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T01:51:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T01:51:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T01:51:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T01:51:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T01:51:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T01:51:27.749 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:51:27.774 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:51:27.877 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:51:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 129/160 2024-09-05T01:51:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:51:28.100 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:51:28.101 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:28.101 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T01:51:28.101 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:51:28.101 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:51:28.101 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:28.152 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:51:28.163 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:51:28.260 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 132/160 2024-09-05T01:51:28.308 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 133/160 2024-09-05T01:51:29.224 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T01:51:29.255 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:51:29.278 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:51:29.279 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:29.279 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T01:51:29.279 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:51:29.279 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:51:29.279 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 1/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 3/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 4/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 5/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 8/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 9/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 10/160 2024-09-05T01:51:29.321 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 11/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 13/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 14/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 15/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 16/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 17/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 18/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 19/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 20/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 21/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 22/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 23/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 24/160 2024-09-05T01:51:29.322 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 25/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 26/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 27/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 28/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 29/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 30/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 31/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 32/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 33/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 35/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 36/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 37/160 2024-09-05T01:51:29.323 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 38/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 39/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T01:51:29.324 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T01:51:29.325 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T01:51:29.326 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T01:51:29.327 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T01:51:29.328 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T01:51:29.329 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T01:51:29.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T01:51:29.331 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 149/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 151/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T01:51:29.332 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T01:51:29.333 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T01:51:29.333 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T01:51:29.333 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T01:51:29.333 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:29.333 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T01:51:29.333 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:51:29.345 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:51:29.345 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:29.345 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T01:51:29.346 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:29.398 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:51:29.422 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:51:29.422 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:29.422 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T01:51:29.422 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:51:29.422 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:51:29.422 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:29.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T01:51:29.469 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:29.469 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-09-05T01:51:29.469 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:29.469 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:29.469 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:29.469 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.470 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.471 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T01:51:29.472 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T01:51:29.473 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T01:51:29.474 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T01:51:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T01:51:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T01:51:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T01:51:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T01:51:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T01:51:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T01:51:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T01:51:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T01:51:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T01:51:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T01:51:29.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T01:51:29.478 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T01:51:29.479 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T01:51:29.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T01:51:29.481 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:51:29.482 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-09-05T01:51:29.724 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:51:29.747 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:51:29.747 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:51:29.747 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T01:51:29.747 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:51:29.747 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:51:29.748 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:29.889 DEBUG:teuthology.parallel:result is None 2024-09-05T01:51:31.945 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T01:51:31.945 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:31.945 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-09-05T01:51:31.945 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.946 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.947 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T01:51:31.948 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T01:51:31.949 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T01:51:31.950 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T01:51:31.951 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T01:51:31.952 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T01:51:31.953 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T01:51:31.954 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T01:51:31.956 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T01:51:31.957 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T01:51:31.958 INFO:teuthology.orchestra.run.smithi139.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T01:51:31.958 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T01:51:31.958 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T01:51:31.958 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T01:51:31.958 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:51:31.958 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-05T01:51:32.347 DEBUG:teuthology.parallel:result is None 2024-09-05T01:51:34.303 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 139/160 2024-09-05T01:51:34.344 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 140/160 2024-09-05T01:51:34.386 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 141/160 2024-09-05T01:51:34.426 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T01:51:34.453 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T01:51:34.639 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 144/160 2024-09-05T01:51:34.684 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T01:51:34.731 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 146/160 2024-09-05T01:51:34.769 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T01:51:34.815 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T01:51:35.382 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:51:35.422 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:51:35.442 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:51:36.288 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 150/160 2024-09-05T01:51:36.386 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T01:51:36.427 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T01:51:36.508 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T01:51:36.539 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T01:51:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T01:51:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:51:36.649 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:51:36.674 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T01:51:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:36.860 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:36.860 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:51:36.952 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:51:36.952 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:51:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:51:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 1/160 2024-09-05T01:51:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2/160 2024-09-05T01:51:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 3/160 2024-09-05T01:51:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 4/160 2024-09-05T01:51:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 5/160 2024-09-05T01:51:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:51:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 8/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 9/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 10/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 11/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 13/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 14/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 15/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 16/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 17/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 18/160 2024-09-05T01:51:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 19/160 2024-09-05T01:51:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 20/160 2024-09-05T01:51:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 21/160 2024-09-05T01:51:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 22/160 2024-09-05T01:51:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 23/160 2024-09-05T01:51:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 24/160 2024-09-05T01:51:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 25/160 2024-09-05T01:51:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 26/160 2024-09-05T01:51:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 27/160 2024-09-05T01:51:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 28/160 2024-09-05T01:51:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 29/160 2024-09-05T01:51:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 30/160 2024-09-05T01:51:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 31/160 2024-09-05T01:51:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 32/160 2024-09-05T01:51:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 33/160 2024-09-05T01:51:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:51:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 35/160 2024-09-05T01:51:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 36/160 2024-09-05T01:51:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 37/160 2024-09-05T01:51:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 38/160 2024-09-05T01:51:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 39/160 2024-09-05T01:51:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T01:51:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T01:51:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T01:51:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T01:51:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T01:51:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T01:51:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T01:51:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T01:51:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T01:51:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T01:51:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T01:51:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T01:51:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T01:51:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T01:51:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T01:51:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T01:51:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T01:51:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T01:51:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T01:51:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T01:51:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T01:51:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T01:51:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T01:51:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T01:51:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T01:51:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T01:51:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T01:51:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T01:51:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T01:51:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T01:51:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T01:51:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T01:51:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T01:51:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T01:51:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T01:51:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T01:51:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T01:51:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T01:51:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T01:51:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T01:51:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T01:51:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T01:51:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T01:51:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T01:51:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T01:51:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T01:51:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T01:51:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T01:51:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T01:51:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T01:51:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T01:51:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T01:51:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T01:51:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T01:51:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T01:51:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T01:51:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T01:51:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T01:51:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T01:51:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T01:51:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T01:51:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T01:51:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T01:51:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T01:51:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T01:51:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T01:51:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T01:51:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T01:51:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T01:51:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T01:51:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T01:51:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T01:51:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T01:51:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T01:51:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T01:51:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T01:51:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T01:51:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T01:51:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 149/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 151/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T01:51:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T01:51:43.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T01:51:43.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:51:43.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T01:51:45.751 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T01:51:45.751 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:45.751 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-09-05T01:51:45.751 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:45.751 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:45.751 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:45.751 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T01:51:45.751 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.752 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.753 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T01:51:45.754 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.755 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T01:51:45.756 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.757 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T01:51:45.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T01:51:45.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T01:51:45.760 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T01:51:45.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T01:51:45.762 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T01:51:45.763 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T01:51:45.763 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T01:51:45.763 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T01:51:45.763 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:51:45.763 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-05T01:51:46.129 DEBUG:teuthology.parallel:result is None 2024-09-05T01:51:46.130 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:51:46.278 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T01:51:46.300 INFO:teuthology.orchestra.run.smithi103.stdout:19.1.1-259.g91c8a7e5.el9 2024-09-05T01:51:46.300 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-259.g91c8a7e5.el9 2024-09-05T01:51:46.301 INFO:teuthology.task.install:The correct ceph version 19.1.1-259.g91c8a7e5 is installed. 2024-09-05T01:51:46.303 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:51:46.443 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T01:51:46.470 INFO:teuthology.orchestra.run.smithi139.stdout:19.1.1-259.g91c8a7e5.el9 2024-09-05T01:51:46.470 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-259.g91c8a7e5.el9 2024-09-05T01:51:46.470 INFO:teuthology.task.install:The correct ceph version 19.1.1-259.g91c8a7e5 is installed. 2024-09-05T01:51:46.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:51:46.620 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T01:51:46.648 INFO:teuthology.orchestra.run.smithi167.stdout:19.1.1-259.g91c8a7e5.el9 2024-09-05T01:51:46.649 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-259.g91c8a7e5.el9 2024-09-05T01:51:46.649 INFO:teuthology.task.install:The correct ceph version 19.1.1-259.g91c8a7e5 is installed. 2024-09-05T01:51:46.652 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-05T01:51:46.652 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:51:46.653 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T01:51:46.681 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:51:46.682 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T01:51:46.716 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:51:46.717 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T01:51:46.749 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-05T01:51:46.749 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:51:46.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T01:51:46.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T01:51:46.842 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:51:46.842 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T01:51:46.871 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T01:51:46.940 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:51:46.940 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T01:51:46.971 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T01:51:47.039 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-05T01:51:47.040 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:51:47.040 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T01:51:47.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T01:51:47.129 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:51:47.130 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T01:51:47.157 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T01:51:47.222 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:51:47.223 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T01:51:47.248 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T01:51:47.314 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-05T01:51:47.314 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:51:47.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T01:51:47.339 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T01:51:47.404 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:51:47.404 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T01:51:47.429 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T01:51:47.495 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:51:47.495 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T01:51:47.526 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T01:51:47.594 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-05T01:51:47.740 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-05T01:51:47.740 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:51:47.741 INFO:tasks.cephadm:Cluster fsid is 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:51:47.741 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-05T01:51:47.741 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.103', 'mon.b': '172.21.15.139', 'mon.c': '172.21.15.167'} 2024-09-05T01:51:47.741 INFO:tasks.cephadm:First mon is mon.a on smithi103 2024-09-05T01:51:47.741 INFO:tasks.cephadm:First mgr is x 2024-09-05T01:51:47.741 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-05T01:51:47.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-09-05T01:51:47.767 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname $(hostname -s) 2024-09-05T01:51:47.793 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-09-05T01:51:47.818 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-05T01:51:47.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:51:47.958 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56000/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-259-g91c8a7e5', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-259.g91c8a7e5'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-04 16:53:37.418684', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-05T01:51:48.095 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:51:48.096 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm 2024-09-05T01:51:48.096 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm 2024-09-05T01:51:48.096 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T01:51:48.489 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 784521 Sep 5 01:51 /home/ubuntu/cephtest/cephadm 2024-09-05T01:51:48.490 DEBUG:teuthology.orchestra.run.smithi139:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T01:51:48.880 INFO:teuthology.orchestra.run.smithi139.stdout:-rw-r--r--. 1 ubuntu ubuntu 784521 Sep 5 01:51 /home/ubuntu/cephtest/cephadm 2024-09-05T01:51:48.881 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T01:51:49.268 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 784521 Sep 5 01:51 /home/ubuntu/cephtest/cephadm 2024-09-05T01:51:49.268 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T01:51:49.288 DEBUG:teuthology.orchestra.run.smithi139:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T01:51:49.306 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T01:51:49.344 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 on all hosts... 2024-09-05T01:51:49.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 pull 2024-09-05T01:51:49.347 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 pull 2024-09-05T01:51:49.350 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 pull 2024-09-05T01:51:50.825 INFO:teuthology.orchestra.run.smithi139.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:51:50.908 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:51:50.999 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:52:19.889 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-05T01:52:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T01:52:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: "image_id": "77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac", 2024-09-05T01:52:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: "repo_digests": [ 2024-09-05T01:52:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067" 2024-09-05T01:52:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-05T01:52:19.890 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-05T01:52:20.275 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-05T01:52:20.275 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T01:52:20.275 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac", 2024-09-05T01:52:20.275 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-09-05T01:52:20.275 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067" 2024-09-05T01:52:20.275 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-05T01:52:20.275 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-05T01:52:20.287 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T01:52:20.287 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T01:52:20.287 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac", 2024-09-05T01:52:20.287 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-09-05T01:52:20.287 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067" 2024-09-05T01:52:20.287 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T01:52:20.287 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T01:52:20.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-09-05T01:52:20.352 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph 2024-09-05T01:52:20.384 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-09-05T01:52:20.411 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-09-05T01:52:20.436 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /etc/ceph 2024-09-05T01:52:20.465 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-09-05T01:52:20.490 INFO:tasks.cephadm:Writing seed config... 2024-09-05T01:52:20.491 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-05T01:52:20.491 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-05T01:52:20.491 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-05T01:52:20.491 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-05T01:52:20.491 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-05T01:52:20.491 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-05T01:52:20.492 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-05T01:52:20.493 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-05T01:52:20.494 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-05T01:52:20.495 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-05T01:52:20.495 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-05T01:52:20.495 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-05T01:52:20.495 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-05T01:52:20.496 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:52:20.496 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-05T01:52:20.511 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 695e8e42-6b29-11ef-bcd6-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-05T01:52:20.511 DEBUG:teuthology.orchestra.run.smithi103:mon.a> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.a.service 2024-09-05T01:52:20.554 DEBUG:teuthology.orchestra.run.smithi103:mgr.x> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.x.service 2024-09-05T01:52:20.597 INFO:tasks.cephadm:Bootstrapping... 2024-09-05T01:52:20.597 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 -v bootstrap --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.103 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:52:20.797 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:52:20.798 INFO:teuthology.orchestra.run.smithi103.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', '-v', 'bootstrap', '--fsid', '695e8e42-6b29-11ef-bcd6-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.103', '--skip-admin-label'] 2024-09-05T01:52:20.798 INFO:teuthology.orchestra.run.smithi103.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-05T01:52:20.798 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying podman|docker is present... 2024-09-05T01:52:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-05T01:52:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying lvm2 is present... 2024-09-05T01:52:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying time synchronization is in place... 2024-09-05T01:52:20.831 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T01:52:20.831 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T01:52:20.837 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T01:52:20.837 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-05T01:52:20.844 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-05T01:52:20.850 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-05T01:52:20.850 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-05T01:52:20.851 INFO:teuthology.orchestra.run.smithi103.stdout:Repeating the final host check... 2024-09-05T01:52:20.872 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-05T01:52:20.873 INFO:teuthology.orchestra.run.smithi103.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-05T01:52:20.873 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl is present 2024-09-05T01:52:20.873 INFO:teuthology.orchestra.run.smithi103.stdout:lvcreate is present 2024-09-05T01:52:20.879 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T01:52:20.879 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T01:52:20.885 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T01:52:20.885 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-05T01:52:20.892 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-05T01:52:20.898 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-05T01:52:20.898 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-05T01:52:20.898 INFO:teuthology.orchestra.run.smithi103.stdout:Host looks OK 2024-09-05T01:52:20.898 INFO:teuthology.orchestra.run.smithi103.stdout:Cluster fsid: 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:20.898 INFO:teuthology.orchestra.run.smithi103.stdout:Acquiring lock 140476726748160 on /run/cephadm/695e8e42-6b29-11ef-bcd6-c7b262605968.lock 2024-09-05T01:52:20.899 INFO:teuthology.orchestra.run.smithi103.stdout:Lock 140476726748160 acquired on /run/cephadm/695e8e42-6b29-11ef-bcd6-c7b262605968.lock 2024-09-05T01:52:20.899 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 3300 ... 2024-09-05T01:52:20.899 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 6789 ... 2024-09-05T01:52:20.899 INFO:teuthology.orchestra.run.smithi103.stdout:Base mon IP(s) is [172.21.15.103:3300, 172.21.15.103:6789], mon addrv is [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-05T01:52:20.902 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.103 metric 100 2024-09-05T01:52:20.903 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.103 metric 100 2024-09-05T01:52:20.905 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-05T01:52:20.905 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-05T01:52:20.905 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-05T01:52:20.907 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-05T01:52:20.908 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-05T01:52:20.908 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T01:52:20.908 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-05T01:52:20.908 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-05T01:52:20.908 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T01:52:20.909 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-05T01:52:20.909 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-05T01:52:20.909 INFO:teuthology.orchestra.run.smithi103.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-05T01:52:20.909 INFO:teuthology.orchestra.run.smithi103.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-05T01:52:20.910 INFO:teuthology.orchestra.run.smithi103.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:52:21.407 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac 2024-09-05T01:52:21.407 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:52:21.407 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Getting image source signatures 2024-09-05T01:52:21.407 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:3f43494a24a2d1fe0d791eb19b05391e70ab9f54611bc9deae80cf7c3fb2dba9 2024-09-05T01:52:21.407 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-05T01:52:21.407 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying config sha256:77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac 2024-09-05T01:52:21.407 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-05T01:52:22.268 INFO:teuthology.orchestra.run.smithi103.stdout:ceph: stdout ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable) 2024-09-05T01:52:22.268 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph version: ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable) 2024-09-05T01:52:22.268 INFO:teuthology.orchestra.run.smithi103.stdout:Extracting ceph user uid/gid from container image... 2024-09-05T01:52:23.157 INFO:teuthology.orchestra.run.smithi103.stdout:stat: stdout 167 167 2024-09-05T01:52:23.157 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial keys... 2024-09-05T01:52:23.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQDXDtlmeaOdHBAAAYj78uDweoy782ANXMsiDg== 2024-09-05T01:52:24.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQDYDtlmIzQIEhAA25aqekgZNrPCUgBz1a1YBw== 2024-09-05T01:52:25.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQDZDtlmoLXiEBAAjTrourkMfURX9AkhKOPUCg== 2024-09-05T01:52:25.882 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial monmap... 2024-09-05T01:52:26.755 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T01:52:26.756 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-05T01:52:26.756 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:26.756 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T01:52:26.756 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool for a [v2:172.21.15.103:3300,v1:172.21.15.103:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T01:52:26.756 INFO:teuthology.orchestra.run.smithi103.stdout:setting min_mon_release = quincy 2024-09-05T01:52:26.756 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: set fsid to 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:26.756 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T01:52:26.756 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:52:26.756 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mon... 2024-09-05T01:52:27.660 INFO:teuthology.orchestra.run.smithi103.stdout:create mon.a on 2024-09-05T01:52:27.856 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-05T01:52:27.991 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-05T01:52:28.167 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968.target. 2024-09-05T01:52:28.167 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968.target. 2024-09-05T01:52:28.411 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.a 2024-09-05T01:52:28.411 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.a.service: Unit ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.a.service not loaded. 2024-09-05T01:52:28.579 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968.target.wants/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.a.service → /etc/systemd/system/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@.service. 2024-09-05T01:52:29.079 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:29 smithi103 ceph-mon[30533]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T01:52:29.079 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:29 smithi103 bash[30512]: 3f1cbad75080b854043880e823d3b28a0c50752a6601fc388242f83d39c43fcb 2024-09-05T01:52:29.094 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T01:52:29.094 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-05T01:52:29.101 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T01:52:29.102 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-05T01:52:29.102 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-05T01:52:29.102 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T01:52:29.102 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon to start... 2024-09-05T01:52:29.102 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon... 2024-09-05T01:52:29.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:29 smithi103 systemd[1]: Started Ceph mon.a for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:52:30.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T01:52:30.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: monmap epoch 1 2024-09-05T01:52:30.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:30.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: last_changed 2024-09-05T01:52:26.169862+0000 2024-09-05T01:52:30.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: created 2024-09-05T01:52:26.169862+0000 2024-09-05T01:52:30.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: min_mon_release 19 (squid) 2024-09-05T01:52:30.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: election_strategy: 1 2024-09-05T01:52:30.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-05T01:52:30.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: fsmap 2024-09-05T01:52:30.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: osdmap e1: 0 total, 0 up, 0 in 2024-09-05T01:52:30.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:30 smithi103 ceph-mon[30533]: mgrmap e1: no daemons active 2024-09-05T01:52:31.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:31 smithi103 ceph-mon[30533]: from='client.? 172.21.15.103:0/3270435142' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-05T01:52:38.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout cluster: 2024-09-05T01:52:38.442 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout id: 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:38.442 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-05T01:52:38.442 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:38.442 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout services: 2024-09-05T01:52:38.442 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 2s) 2024-09-05T01:52:38.442 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout data: 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pgs: 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.506+0000 7f57fe3e9640 1 Processor -- start 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.506+0000 7f57fe3e9640 1 -- start start 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.506+0000 7f57fe3e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f8107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.506+0000 7f57fe3e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8107e10 con 0x7f57f8107440 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.507+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f8107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:38.443 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.507+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f8107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36940/0 (socket says 172.21.15.103:36940) 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.507+0000 7f57f7fff640 1 -- 172.21.15.103:0/3351383186 learned_addr learned my addr 172.21.15.103:0/3351383186 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.507+0000 7f57f7fff640 1 -- 172.21.15.103:0/3351383186 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f8108640 con 0x7f57f8107440 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.509+0000 7f57f7fff640 1 --2- 172.21.15.103:0/3351383186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f8107840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009820 tx=0x7f57e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1196e6334d95477 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.510+0000 7f57f6ffd640 1 -- 172.21.15.103:0/3351383186 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57e8037070 con 0x7f57f8107440 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.510+0000 7f57f6ffd640 1 -- 172.21.15.103:0/3351383186 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e8032440 con 0x7f57f8107440 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.510+0000 7f57f6ffd640 1 -- 172.21.15.103:0/3351383186 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57e8032740 con 0x7f57f8107440 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.510+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3351383186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 msgr2=0x7f57f8107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.510+0000 7f57fe3e9640 1 --2- 172.21.15.103:0/3351383186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f8107840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009820 tx=0x7f57e802a270 comp rx=0 tx=0).stop 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.510+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3351383186 shutdown_connections 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.510+0000 7f57fe3e9640 1 --2- 172.21.15.103:0/3351383186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f8107840 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.510+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3351383186 >> 172.21.15.103:0/3351383186 conn(0x7f57f8102bb0 msgr2=0x7f57f8105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:38.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.511+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3351383186 shutdown_connections 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.511+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3351383186 wait complete. 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.511+0000 7f57fe3e9640 1 Processor -- start 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.511+0000 7f57fe3e9640 1 -- start start 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57fe3e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57fe3e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8108be0 con 0x7f57f8107440 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36942/0 (socket says 172.21.15.103:36942) 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57f7fff640 1 -- 172.21.15.103:0/3270435142 learned_addr learned my addr 172.21.15.103:0/3270435142 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57f7fff640 1 -- 172.21.15.103:0/3270435142 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f819b030 con 0x7f57f8107440 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57f7fff640 1 --2- 172.21.15.103:0/3270435142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f819aaf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f57e80090b0 tx=0x7f57e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:38.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57f57fa640 1 -- 172.21.15.103:0/3270435142 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57e8041070 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57f57fa640 1 -- 172.21.15.103:0/3270435142 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e803c7a0 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57f57fa640 1 -- 172.21.15.103:0/3270435142 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57e8037040 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3270435142 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f819b2c0 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.512+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3270435142 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f819dfb0 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.513+0000 7f57f57fa640 1 -- 172.21.15.103:0/3270435142 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f57e8044070 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.513+0000 7f57f57fa640 1 -- 172.21.15.103:0/3270435142 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f57e803d3f0 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.513+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3270435142 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f8109230 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.516+0000 7f57f57fa640 1 -- 172.21.15.103:0/3270435142 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f57e8046450 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.576+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3270435142 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f57f802d910 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.576+0000 7f57f57fa640 1 -- 172.21.15.103:0/3270435142 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7f57e8046630 con 0x7f57f8107440 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.577+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3270435142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 msgr2=0x7f57f819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.577+0000 7f57fe3e9640 1 --2- 172.21.15.103:0/3270435142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f819aaf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f57e80090b0 tx=0x7f57e80046c0 comp rx=0 tx=0).stop 2024-09-05T01:52:38.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.577+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3270435142 shutdown_connections 2024-09-05T01:52:38.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.577+0000 7f57fe3e9640 1 --2- 172.21.15.103:0/3270435142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f8107440 0x7f57f819aaf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:38.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.577+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3270435142 >> 172.21.15.103:0/3270435142 conn(0x7f57f8102bb0 msgr2=0x7f57f818edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:38.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.577+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3270435142 shutdown_connections 2024-09-05T01:52:38.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:31.577+0000 7f57fe3e9640 1 -- 172.21.15.103:0/3270435142 wait complete. 2024-09-05T01:52:38.447 INFO:teuthology.orchestra.run.smithi103.stdout:mon is available 2024-09-05T01:52:38.447 INFO:teuthology.orchestra.run.smithi103.stdout:Assimilating anything we can from ceph.conf... 2024-09-05T01:52:41.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:41 smithi103 ceph-mon[30533]: from='client.? 172.21.15.103:0/1929650117' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-05T01:52:41.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:41 smithi103 ceph-mon[30533]: from='client.? 172.21.15.103:0/1929650117' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-05T01:52:42.029 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:42.029 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T01:52:42.029 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T01:52:42.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.388+0000 7f80f8e5a640 1 Processor -- start 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.388+0000 7f80f8e5a640 1 -- start start 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.388+0000 7f80f8e5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.388+0000 7f80f8e5a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80f407c870 con 0x7f80f407de40 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.388+0000 7f80f259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.389+0000 7f80f259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f407c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56490/0 (socket says 172.21.15.103:56490) 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.389+0000 7f80f259c640 1 -- 172.21.15.103:0/3356250949 learned_addr learned my addr 172.21.15.103:0/3356250949 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.389+0000 7f80f259c640 1 -- 172.21.15.103:0/3356250949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f407c9f0 con 0x7f80f407de40 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f259c640 1 --2- 172.21.15.103:0/3356250949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f407c2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009820 tx=0x7f80e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45ab8f04edff155 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:42.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f1d9b640 1 -- 172.21.15.103:0/3356250949 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e4037070 con 0x7f80f407de40 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f1d9b640 1 -- 172.21.15.103:0/3356250949 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e4032440 con 0x7f80f407de40 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f1d9b640 1 -- 172.21.15.103:0/3356250949 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e4032740 con 0x7f80f407de40 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f8e5a640 1 -- 172.21.15.103:0/3356250949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 msgr2=0x7f80f407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f8e5a640 1 --2- 172.21.15.103:0/3356250949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f407c2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009820 tx=0x7f80e402a270 comp rx=0 tx=0).stop 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f8e5a640 1 -- 172.21.15.103:0/3356250949 shutdown_connections 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f8e5a640 1 --2- 172.21.15.103:0/3356250949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f407c2a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f8e5a640 1 -- 172.21.15.103:0/3356250949 >> 172.21.15.103:0/3356250949 conn(0x7f80f4102bc0 msgr2=0x7f80f4105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f8e5a640 1 -- 172.21.15.103:0/3356250949 shutdown_connections 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.390+0000 7f80f8e5a640 1 -- 172.21.15.103:0/3356250949 wait complete. 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.391+0000 7f80f8e5a640 1 Processor -- start 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.391+0000 7f80f8e5a640 1 -- start start 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.391+0000 7f80f8e5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:42.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.391+0000 7f80f8e5a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80f407cf90 con 0x7f80f407de40 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80f259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80f259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56498/0 (socket says 172.21.15.103:56498) 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80f259c640 1 -- 172.21.15.103:0/1929650117 learned_addr learned my addr 172.21.15.103:0/1929650117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80f259c640 1 -- 172.21.15.103:0/1929650117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f419d230 con 0x7f80f407de40 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80f259c640 1 --2- 172.21.15.103:0/1929650117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f419ccf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f80e40090b0 tx=0x7f80e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80eb7fe640 1 -- 172.21.15.103:0/1929650117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e4041070 con 0x7f80f407de40 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80eb7fe640 1 -- 172.21.15.103:0/1929650117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e403c7a0 con 0x7f80f407de40 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80eb7fe640 1 -- 172.21.15.103:0/1929650117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e4037040 con 0x7f80f407de40 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80f8e5a640 1 -- 172.21.15.103:0/1929650117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f419d4c0 con 0x7f80f407de40 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80f8e5a640 1 -- 172.21.15.103:0/1929650117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f41a01a0 con 0x7f80f407de40 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.392+0000 7f80eb7fe640 1 -- 172.21.15.103:0/1929650117 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80e4044070 con 0x7f80f407de40 2024-09-05T01:52:42.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.393+0000 7f80eb7fe640 1 -- 172.21.15.103:0/1929650117 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80e403d3f0 con 0x7f80f407de40 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.393+0000 7f80f8e5a640 1 -- 172.21.15.103:0/1929650117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f407c320 con 0x7f80f407de40 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.396+0000 7f80eb7fe640 1 -- 172.21.15.103:0/1929650117 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f80e4046450 con 0x7f80f407de40 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.453+0000 7f80f8e5a640 1 -- 172.21.15.103:0/1929650117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f80f41a1900 con 0x7f80f407de40 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.456+0000 7f80eb7fe640 1 -- 172.21.15.103:0/1929650117 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f80e4046630 con 0x7f80f407de40 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.457+0000 7f80f8e5a640 1 -- 172.21.15.103:0/1929650117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 msgr2=0x7f80f419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.457+0000 7f80f8e5a640 1 --2- 172.21.15.103:0/1929650117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f419ccf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f80e40090b0 tx=0x7f80e40046c0 comp rx=0 tx=0).stop 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.457+0000 7f80f8e5a640 1 -- 172.21.15.103:0/1929650117 shutdown_connections 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.457+0000 7f80f8e5a640 1 --2- 172.21.15.103:0/1929650117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f407de40 0x7f80f419ccf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.457+0000 7f80f8e5a640 1 -- 172.21.15.103:0/1929650117 >> 172.21.15.103:0/1929650117 conn(0x7f80f4102bc0 msgr2=0x7f80f41039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.457+0000 7f80f8e5a640 1 -- 172.21.15.103:0/1929650117 shutdown_connections 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:41.457+0000 7f80f8e5a640 1 -- 172.21.15.103:0/1929650117 wait complete. 2024-09-05T01:52:42.034 INFO:teuthology.orchestra.run.smithi103.stdout:Generating new minimal ceph.conf... 2024-09-05T01:52:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:42 smithi103 ceph-mon[30533]: from='client.? 172.21.15.103:0/600145854' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:52:42.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.423+0000 7fabc04cd640 1 Processor -- start 2024-09-05T01:52:42.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.423+0000 7fabc04cd640 1 -- start start 2024-09-05T01:52:42.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.423+0000 7fabc04cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb8107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:42.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.424+0000 7fabc04cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8107e10 con 0x7fabb8107440 2024-09-05T01:52:42.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.424+0000 7fabbe269640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb8107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:42.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.424+0000 7fabbe269640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb8107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56504/0 (socket says 172.21.15.103:56504) 2024-09-05T01:52:42.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.424+0000 7fabbe269640 1 -- 172.21.15.103:0/2618049477 learned_addr learned my addr 172.21.15.103:0/2618049477 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.424+0000 7fabbe269640 1 -- 172.21.15.103:0/2618049477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb8108640 con 0x7fabb8107440 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.425+0000 7fabbe269640 1 --2- 172.21.15.103:0/2618049477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb8107840 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009820 tx=0x7fabb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531038cd4fc2efb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.425+0000 7fabbd267640 1 -- 172.21.15.103:0/2618049477 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb4037070 con 0x7fabb8107440 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.425+0000 7fabbd267640 1 -- 172.21.15.103:0/2618049477 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fabb40328d0 con 0x7fabb8107440 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.425+0000 7fabbd267640 1 -- 172.21.15.103:0/2618049477 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb40307d0 con 0x7fabb8107440 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.425+0000 7fabc04cd640 1 -- 172.21.15.103:0/2618049477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 msgr2=0x7fabb8107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.425+0000 7fabc04cd640 1 --2- 172.21.15.103:0/2618049477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb8107840 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009820 tx=0x7fabb402a270 comp rx=0 tx=0).stop 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.425+0000 7fabc04cd640 1 -- 172.21.15.103:0/2618049477 shutdown_connections 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.425+0000 7fabc04cd640 1 --2- 172.21.15.103:0/2618049477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb8107840 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.425+0000 7fabc04cd640 1 -- 172.21.15.103:0/2618049477 >> 172.21.15.103:0/2618049477 conn(0x7fabb8102bb0 msgr2=0x7fabb8105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.426+0000 7fabc04cd640 1 -- 172.21.15.103:0/2618049477 shutdown_connections 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.426+0000 7fabc04cd640 1 -- 172.21.15.103:0/2618049477 wait complete. 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.426+0000 7fabc04cd640 1 Processor -- start 2024-09-05T01:52:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.426+0000 7fabc04cd640 1 -- start start 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.426+0000 7fabc04cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.426+0000 7fabc04cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabb810b9f0 con 0x7fabb8107440 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.426+0000 7fabbe269640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.426+0000 7fabbe269640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb819b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56510/0 (socket says 172.21.15.103:56510) 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.426+0000 7fabbe269640 1 -- 172.21.15.103:0/600145854 learned_addr learned my addr 172.21.15.103:0/600145854 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.427+0000 7fabbe269640 1 -- 172.21.15.103:0/600145854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb819b800 con 0x7fabb8107440 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.427+0000 7fabbe269640 1 --2- 172.21.15.103:0/600145854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb819b2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009940 tx=0x7fabb4032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.427+0000 7fabaf7fe640 1 -- 172.21.15.103:0/600145854 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb4042070 con 0x7fabb8107440 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.427+0000 7fabaf7fe640 1 -- 172.21.15.103:0/600145854 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fabb4030c80 con 0x7fabb8107440 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.427+0000 7fabc04cd640 1 -- 172.21.15.103:0/600145854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb819ba90 con 0x7fabb8107440 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.427+0000 7fabaf7fe640 1 -- 172.21.15.103:0/600145854 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb4037040 con 0x7fabb8107440 2024-09-05T01:52:42.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.427+0000 7fabc04cd640 1 -- 172.21.15.103:0/600145854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb819e780 con 0x7fabb8107440 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.427+0000 7fabaf7fe640 1 -- 172.21.15.103:0/600145854 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fabb4045070 con 0x7fabb8107440 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.427+0000 7fabaf7fe640 1 -- 172.21.15.103:0/600145854 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fabb40479e0 con 0x7fabb8107440 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.428+0000 7fabc04cd640 1 -- 172.21.15.103:0/600145854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb810bb70 con 0x7fabb8107440 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.430+0000 7fabaf7fe640 1 -- 172.21.15.103:0/600145854 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fabb403c5f0 con 0x7fabb8107440 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.487+0000 7fabc04cd640 1 -- 172.21.15.103:0/600145854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fabb819ed10 con 0x7fabb8107440 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.487+0000 7fabaf7fe640 1 -- 172.21.15.103:0/600145854 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fabb4047be0 con 0x7fabb8107440 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.488+0000 7fabc04cd640 1 -- 172.21.15.103:0/600145854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 msgr2=0x7fabb819b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.488+0000 7fabc04cd640 1 --2- 172.21.15.103:0/600145854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb819b2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009940 tx=0x7fabb4032e10 comp rx=0 tx=0).stop 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.488+0000 7fabc04cd640 1 -- 172.21.15.103:0/600145854 shutdown_connections 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.488+0000 7fabc04cd640 1 --2- 172.21.15.103:0/600145854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8107440 0x7fabb819b2c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.488+0000 7fabc04cd640 1 -- 172.21.15.103:0/600145854 >> 172.21.15.103:0/600145854 conn(0x7fabb8102bb0 msgr2=0x7fabb81035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.488+0000 7fabc04cd640 1 -- 172.21.15.103:0/600145854 shutdown_connections 2024-09-05T01:52:42.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:42.488+0000 7fabc04cd640 1 -- 172.21.15.103:0/600145854 wait complete. 2024-09-05T01:52:42.997 INFO:teuthology.orchestra.run.smithi103.stdout:Restarting the monitor... 2024-09-05T01:52:43.264 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:43 smithi103 systemd[1]: Stopping Ceph mon.a for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:52:43.519 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:43 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a[30529]: 2024-09-05T01:52:43.263+0000 7f6e70d87640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:52:43.519 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:43 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a[30529]: 2024-09-05T01:52:43.263+0000 7f6e70d87640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-05T01:52:43.771 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:43 smithi103 podman[30775]: 2024-09-05 01:52:43.51901167 +0000 UTC m=+0.367002328 container died 3f1cbad75080b854043880e823d3b28a0c50752a6601fc388242f83d39c43fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-05T01:52:44.105 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:43 smithi103 podman[30775]: 2024-09-05 01:52:43.879387199 +0000 UTC m=+0.727377863 container cleanup 3f1cbad75080b854043880e823d3b28a0c50752a6601fc388242f83d39c43fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-05T01:52:44.106 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:43 smithi103 bash[30775]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a 2024-09-05T01:52:44.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 podman[30787]: 2024-09-05 01:52:44.105245276 +0000 UTC m=+0.589943075 container remove 3f1cbad75080b854043880e823d3b28a0c50752a6601fc388242f83d39c43fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-91c8a7e) 2024-09-05T01:52:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-05T01:52:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 systemd[1]: Stopped Ceph mon.a for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:52:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 systemd[1]: Starting Ceph mon.a for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:52:44.716 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 podman[30883]: 2024-09-05 01:52:44.44785733 +0000 UTC m=+0.103612330 container create 462eb3fe00994516fc228e9775e4f98a2948f15d21be2bc309fe21c85399dddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2) 2024-09-05T01:52:44.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 podman[30883]: 2024-09-05 01:52:44.36026427 +0000 UTC m=+0.016019279 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:52:44.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 podman[30883]: 2024-09-05 01:52:44.692033224 +0000 UTC m=+0.347788223 container init 462eb3fe00994516fc228e9775e4f98a2948f15d21be2bc309fe21c85399dddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:52:44.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 podman[30883]: 2024-09-05 01:52:44.695063369 +0000 UTC m=+0.350818370 container start 462eb3fe00994516fc228e9775e4f98a2948f15d21be2bc309fe21c85399dddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-05T01:52:44.805 INFO:teuthology.orchestra.run.smithi103.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-05T01:52:45.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: set uid:gid to 167:167 (ceph:ceph) 2024-09-05T01:52:45.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable), process ceph-mon, pid 2 2024-09-05T01:52:45.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: pidfile_write: ignore empty --pid-file 2024-09-05T01:52:45.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: load: jerasure load: lrc 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: RocksDB version: 7.9.2 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Git sha 0 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Compile date 2024-09-04 15:32:03 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: DB SUMMARY 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: DB Session ID: WUNW5PWI9I2MLPGHH60T 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: CURRENT file: CURRENT 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: IDENTITY file: IDENTITY 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104571 ; 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.error_if_exists: 0 2024-09-05T01:52:45.013 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.create_if_missing: 0 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.paranoid_checks: 1 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.env: 0x55f395591940 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.fs: PosixFileSystem 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.info_log: 0x55f3968c36e0 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_file_opening_threads: 16 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.statistics: (nil) 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.use_fsync: 0 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_log_file_size: 0 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-05T01:52:45.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.keep_log_file_num: 1000 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.recycle_log_file_num: 0 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.allow_fallocate: 1 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.allow_mmap_reads: 0 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.allow_mmap_writes: 0 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.use_direct_reads: 0 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.create_missing_column_families: 0 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.db_log_dir: 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.wal_dir: 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-05T01:52:45.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.advise_random_on_open: 1 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.db_write_buffer_size: 0 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.write_buffer_manager: 0x55f3968d2dc0 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.rate_limiter: (nil) 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.wal_recovery_mode: 2 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.enable_thread_tracking: 0 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.enable_pipelined_write: 0 2024-09-05T01:52:45.016 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.unordered_write: 0 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.row_cache: None 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.wal_filter: None 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.allow_ingest_behind: 0 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.two_write_queues: 0 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.manual_wal_flush: 0 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.wal_compression: 0 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.atomic_flush: 0 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-05T01:52:45.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-05T01:52:45.018 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-05T01:52:45.018 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.log_readahead_size: 0 2024-09-05T01:52:45.018 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-05T01:52:45.018 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.best_efforts_recovery: 0 2024-09-05T01:52:45.018 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-05T01:52:45.018 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-05T01:52:45.018 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.allow_data_in_errors: 0 2024-09-05T01:52:45.018 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.db_host_id: __hostname__ 2024-09-05T01:52:45.018 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-05T01:52:45.019 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_background_jobs: 2 2024-09-05T01:52:45.019 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_background_compactions: -1 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_subcompactions: 1 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_total_wal_size: 0 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_open_files: -1 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bytes_per_sync: 0 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-05T01:52:45.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_readahead_size: 0 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_background_flushes: -1 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Compression algorithms supported: 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: kZSTD supported: 0 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: kXpressCompression supported: 0 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: kBZip2Compression supported: 0 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: kLZ4Compression supported: 1 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: kZlibCompression supported: 1 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: kLZ4HCCompression supported: 1 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: kSnappyCompression supported: 1 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-05T01:52:45.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.merge_operator: 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_filter: None 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_filter_factory: None 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.sst_partitioner_factory: None 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f3968c3320) 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks: 1 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_top_level_index_and_filter: 1 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_type: 0 2024-09-05T01:52:45.022 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_index_type: 0 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_shortening: 1 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: checksum: 4 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: no_block_cache: 0 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache: 0x55f3968b71f0 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_name: BinnedLRUCache 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_options: 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: capacity : 536870912 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_shard_bits : 4 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: strict_capacity_limit : 0 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: high_pri_pool_ratio: 0.000 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_compressed: (nil) 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: persistent_cache: (nil) 2024-09-05T01:52:45.023 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size: 4096 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size_deviation: 10 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_restart_interval: 16 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_block_restart_interval: 1 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: metadata_block_size: 4096 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: partition_filters: 0 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: use_delta_encoding: 1 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: filter_policy: bloomfilter 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: whole_key_filtering: 1 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: verify_compression: 0 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: read_amp_bytes_per_bit: 0 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: format_version: 5 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_index_compression: 1 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_align: 0 2024-09-05T01:52:45.024 INFO:journalctl@ceph.mon.a.smithi103.stdout: max_auto_readahead_size: 262144 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout: prepopulate_block_cache: 0 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout: initial_auto_readahead_size: 8192 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_file_reads_for_auto_readahead: 2 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.write_buffer_size: 33554432 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_write_buffer_number: 2 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression: NoCompression 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression: Disabled 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.prefix_extractor: nullptr 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.num_levels: 7 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-05T01:52:45.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression_opts.level: 32767 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression_opts.strategy: 0 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-05T01:52:45.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression_opts.enabled: false 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.target_file_size_base: 67108864 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-05T01:52:45.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.arena_block_size: 1048576 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.disable_auto_compactions: 0 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-05T01:52:45.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.inplace_update_support: 0 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.bloom_locality: 0 2024-09-05T01:52:45.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.max_successive_merges: 0 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.paranoid_file_checks: 0 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.force_consistency_checks: 1 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.report_bg_io_stats: 0 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.ttl: 2592000 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.enable_blob_files: false 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.min_blob_size: 0 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.blob_file_size: 268435456 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-05T01:52:45.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.blob_file_starting_level: 0 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ea673cf9-1e51-46e3-a4f3-d560b4c81c56 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725501164721170, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-05T01:52:45.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725501164722209, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99658, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 97552, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14076, "raw_average_key_size": 49, "raw_value_size": 89846, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725501164, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ea673cf9-1e51-46e3-a4f3-d560b4c81c56", "db_session_id": "WUNW5PWI9I2MLPGHH60T", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725501164722304, "job": 1, "event": "recovery_finished"} 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f3968e4e00 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: DB pointer 0x55f3969e0000 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** DB Stats ** 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T01:52:45.032 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: L0 2/0 99.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: Sum 2/0 99.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T01:52:45.033 INFO:journalctl@ceph.mon.a.smithi103.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative compaction: 0.00 GB write, 30.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval compaction: 0.00 GB write, 30.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-05T01:52:45.034 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache BinnedLRUCache@0x55f3968b71f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: starting mon.a rank 0 at public addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] at bind addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mon.a@-1(???) e1 preinit fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mon.a@-1(???).mds e1 new map 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mon.a@-1(???).mds e1 print_map 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: e1 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: btime 2024-09-05T01:52:29:043577+0000 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: legacy client fscid: -1 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout: No filesystems configured 2024-09-05T01:52:45.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: monmap epoch 1 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: last_changed 2024-09-05T01:52:26.169862+0000 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: created 2024-09-05T01:52:26.169862+0000 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: min_mon_release 19 (squid) 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: election_strategy: 1 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: fsmap 2024-09-05T01:52:45.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: osdmap e1: 0 total, 0 up, 0 in 2024-09-05T01:52:45.037 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 ceph-mon[30902]: mgrmap e1: no daemons active 2024-09-05T01:52:45.037 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 bash[30883]: 462eb3fe00994516fc228e9775e4f98a2948f15d21be2bc309fe21c85399dddf 2024-09-05T01:52:45.037 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:44 smithi103 systemd[1]: Started Ceph mon.a for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:52:45.872 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.201+0000 7f98f257a640 1 Processor -- start 2024-09-05T01:52:45.872 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.201+0000 7f98f257a640 1 -- start start 2024-09-05T01:52:45.872 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.201+0000 7f98f257a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec107890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.201+0000 7f98f257a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec107e60 con 0x7f98ec107490 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.201+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.201+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56526/0 (socket says 172.21.15.103:56526) 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.201+0000 7f98ebfff640 1 -- 172.21.15.103:0/1484096016 learned_addr learned my addr 172.21.15.103:0/1484096016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.202+0000 7f98ebfff640 1 -- 172.21.15.103:0/1484096016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec108640 con 0x7f98ec107490 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.203+0000 7f98ebfff640 1 --2- 172.21.15.103:0/1484096016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec107890 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009820 tx=0x7f98dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e2938a7afedc9c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98eb7fe640 1 -- 172.21.15.103:0/1484096016 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98dc037070 con 0x7f98ec107490 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98eb7fe640 1 -- 172.21.15.103:0/1484096016 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f98dc0328d0 con 0x7f98ec107490 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98eb7fe640 1 -- 172.21.15.103:0/1484096016 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98dc0307d0 con 0x7f98ec107490 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98f257a640 1 -- 172.21.15.103:0/1484096016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 msgr2=0x7f98ec107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98f257a640 1 --2- 172.21.15.103:0/1484096016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec107890 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009820 tx=0x7f98dc02a270 comp rx=0 tx=0).stop 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98f257a640 1 -- 172.21.15.103:0/1484096016 shutdown_connections 2024-09-05T01:52:45.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98f257a640 1 --2- 172.21.15.103:0/1484096016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec107890 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98f257a640 1 -- 172.21.15.103:0/1484096016 >> 172.21.15.103:0/1484096016 conn(0x7f98ec102be0 msgr2=0x7f98ec105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98f257a640 1 -- 172.21.15.103:0/1484096016 shutdown_connections 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.204+0000 7f98f257a640 1 -- 172.21.15.103:0/1484096016 wait complete. 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.205+0000 7f98f257a640 1 Processor -- start 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.205+0000 7f98f257a640 1 -- start start 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.205+0000 7f98f257a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.205+0000 7f98f257a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec10b9f0 con 0x7f98ec107490 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.205+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.205+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56532/0 (socket says 172.21.15.103:56532) 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.205+0000 7f98ebfff640 1 -- 172.21.15.103:0/104814250 learned_addr learned my addr 172.21.15.103:0/104814250 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.206+0000 7f98ebfff640 1 -- 172.21.15.103:0/104814250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec19b7f0 con 0x7f98ec107490 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.206+0000 7f98ebfff640 1 --2- 172.21.15.103:0/104814250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec19b2b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009940 tx=0x7f98dc032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:45.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.206+0000 7f98e9ffb640 1 -- 172.21.15.103:0/104814250 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98dc042070 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.206+0000 7f98e9ffb640 1 -- 172.21.15.103:0/104814250 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f98dc030c80 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.206+0000 7f98f257a640 1 -- 172.21.15.103:0/104814250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec19ba80 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.206+0000 7f98e9ffb640 1 -- 172.21.15.103:0/104814250 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98dc037040 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.206+0000 7f98f257a640 1 -- 172.21.15.103:0/104814250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec19e770 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.206+0000 7f98e9ffb640 1 -- 172.21.15.103:0/104814250 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f98dc045070 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.206+0000 7f98e9ffb640 1 -- 172.21.15.103:0/104814250 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98dc047a20 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.207+0000 7f98f257a640 1 -- 172.21.15.103:0/104814250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ec107910 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.209+0000 7f98e9ffb640 1 -- 172.21.15.103:0/104814250 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f98dc03c5f0 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.266+0000 7f98f257a640 1 -- 172.21.15.103:0/104814250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f98ec19ed50 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.270+0000 7f98e9ffb640 1 -- 172.21.15.103:0/104814250 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f98dc03c7d0 con 0x7f98ec107490 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.271+0000 7f98f257a640 1 -- 172.21.15.103:0/104814250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 msgr2=0x7f98ec19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.271+0000 7f98f257a640 1 --2- 172.21.15.103:0/104814250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec19b2b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009940 tx=0x7f98dc032e10 comp rx=0 tx=0).stop 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.271+0000 7f98f257a640 1 -- 172.21.15.103:0/104814250 shutdown_connections 2024-09-05T01:52:45.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.271+0000 7f98f257a640 1 --2- 172.21.15.103:0/104814250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98ec107490 0x7f98ec19b2b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:45.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.271+0000 7f98f257a640 1 -- 172.21.15.103:0/104814250 >> 172.21.15.103:0/104814250 conn(0x7f98ec102be0 msgr2=0x7f98ec103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:45.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.271+0000 7f98f257a640 1 -- 172.21.15.103:0/104814250 shutdown_connections 2024-09-05T01:52:45.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:45.271+0000 7f98f257a640 1 -- 172.21.15.103:0/104814250 wait complete. 2024-09-05T01:52:45.876 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-05T01:52:45.898 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:52:45.898 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mgr... 2024-09-05T01:52:45.898 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-05T01:52:45.899 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-05T01:52:46.153 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T01:52:46.153 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.x.service: Unit ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.x.service not loaded. 2024-09-05T01:52:46.305 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968.target.wants/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@.service. 2024-09-05T01:52:46.535 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:46 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/104814250' entity='client.admin' 2024-09-05T01:52:46.536 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:46 smithi103 systemd[1]: Starting Ceph mgr.x for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:52:46.899 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:46 smithi103 podman[31112]: 2024-09-05 01:52:46.4373865 +0000 UTC m=+0.016005258 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:52:46.899 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:46 smithi103 podman[31112]: 2024-09-05 01:52:46.550668109 +0000 UTC m=+0.129286851 container create a70b6e132a430e02244e7888ee331263d78f62ad55da4e2db17b4940e46520fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-05T01:52:48.310 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T01:52:48.310 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-05T01:52:48.318 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T01:52:48.318 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-05T01:52:48.318 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-05T01:52:48.318 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T01:52:48.327 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T01:52:48.327 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-05T01:52:48.334 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T01:52:48.334 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-05T01:52:48.334 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-05T01:52:48.334 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-05T01:52:48.335 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr to start... 2024-09-05T01:52:48.335 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr... 2024-09-05T01:52:48.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 podman[31112]: 2024-09-05 01:52:48.211104256 +0000 UTC m=+1.789722999 container init a70b6e132a430e02244e7888ee331263d78f62ad55da4e2db17b4940e46520fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-05T01:52:48.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 podman[31112]: 2024-09-05 01:52:48.214991434 +0000 UTC m=+1.793610178 container start a70b6e132a430e02244e7888ee331263d78f62ad55da4e2db17b4940e46520fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-05T01:52:48.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 bash[31112]: a70b6e132a430e02244e7888ee331263d78f62ad55da4e2db17b4940e46520fc 2024-09-05T01:52:48.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 systemd[1]: Started Ceph mgr.x for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:52:48.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:48.352+0000 7f299cf52040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:52:48.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:48.411+0000 7f299cf52040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:52:48.778 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:48.562+0000 7f299cf52040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:52:48.778 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:48.633+0000 7f299cf52040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:52:49.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:48 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/871365180' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T01:52:49.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:48.789+0000 7f299cf52040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:52:49.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:48.937+0000 7f299cf52040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:52:49.468 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:49 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:49.260+0000 7f299cf52040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:52:49.469 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:49 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:49.320+0000 7f299cf52040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T01:52:49.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:49.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T01:52:49.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T01:52:49.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T01:52:49.482 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T01:52:49.482 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T01:52:49.482 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T01:52:49.482 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:49.482 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T01:52:49.482 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T01:52:49.482 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T01:52:49.482 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T01:52:29:043577+0000", 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:49.483 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T01:52:29.046314+0000", 2024-09-05T01:52:49.484 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.777+0000 7fee66a80640 1 Processor -- start 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.777+0000 7fee66a80640 1 -- start start 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.777+0000 7fee66a80640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6010a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.777+0000 7fee66a80640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee600757a0 con 0x7fee60107e70 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.777+0000 7fee6481c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6010a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.778+0000 7fee6481c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6010a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56558/0 (socket says 172.21.15.103:56558) 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.778+0000 7fee6481c640 1 -- 172.21.15.103:0/227901113 learned_addr learned my addr 172.21.15.103:0/227901113 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.778+0000 7fee6481c640 1 -- 172.21.15.103:0/227901113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6010a7a0 con 0x7fee60107e70 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.779+0000 7fee6481c640 1 --2- 172.21.15.103:0/227901113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6010a260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fee54009820 tx=0x7fee5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e5e690af713001 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.779+0000 7fee5f7fe640 1 -- 172.21.15.103:0/227901113 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee54037070 con 0x7fee60107e70 2024-09-05T01:52:49.485 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.779+0000 7fee5f7fe640 1 -- 172.21.15.103:0/227901113 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fee540328d0 con 0x7fee60107e70 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.779+0000 7fee5f7fe640 1 -- 172.21.15.103:0/227901113 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee540307d0 con 0x7fee60107e70 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.780+0000 7fee66a80640 1 -- 172.21.15.103:0/227901113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 msgr2=0x7fee6010a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.780+0000 7fee66a80640 1 --2- 172.21.15.103:0/227901113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6010a260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fee54009820 tx=0x7fee5402a270 comp rx=0 tx=0).stop 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.780+0000 7fee66a80640 1 -- 172.21.15.103:0/227901113 shutdown_connections 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.780+0000 7fee66a80640 1 --2- 172.21.15.103:0/227901113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6010a260 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.780+0000 7fee66a80640 1 -- 172.21.15.103:0/227901113 >> 172.21.15.103:0/227901113 conn(0x7fee60101a50 msgr2=0x7fee60103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.780+0000 7fee66a80640 1 -- 172.21.15.103:0/227901113 shutdown_connections 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.780+0000 7fee66a80640 1 -- 172.21.15.103:0/227901113 wait complete. 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.781+0000 7fee66a80640 1 Processor -- start 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.781+0000 7fee66a80640 1 -- start start 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.781+0000 7fee66a80640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.781+0000 7fee66a80640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee6010b7e0 con 0x7fee60107e70 2024-09-05T01:52:49.486 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.781+0000 7fee6481c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.781+0000 7fee6481c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56562/0 (socket says 172.21.15.103:56562) 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.781+0000 7fee6481c640 1 -- 172.21.15.103:0/871365180 learned_addr learned my addr 172.21.15.103:0/871365180 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.781+0000 7fee6481c640 1 -- 172.21.15.103:0/871365180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6019b870 con 0x7fee60107e70 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.781+0000 7fee6481c640 1 --2- 172.21.15.103:0/871365180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6019b330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fee540097f0 tx=0x7fee54030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.782+0000 7fee5dffb640 1 -- 172.21.15.103:0/871365180 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee54037040 con 0x7fee60107e70 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.782+0000 7fee5dffb640 1 -- 172.21.15.103:0/871365180 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fee5403b3f0 con 0x7fee60107e70 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.782+0000 7fee66a80640 1 -- 172.21.15.103:0/871365180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6019bb00 con 0x7fee60107e70 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.782+0000 7fee5dffb640 1 -- 172.21.15.103:0/871365180 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee5403a520 con 0x7fee60107e70 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.782+0000 7fee66a80640 1 -- 172.21.15.103:0/871365180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6019e7f0 con 0x7fee60107e70 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.782+0000 7fee5dffb640 1 -- 172.21.15.103:0/871365180 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee5403a710 con 0x7fee60107e70 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.782+0000 7fee5dffb640 1 -- 172.21.15.103:0/871365180 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee54044b50 con 0x7fee60107e70 2024-09-05T01:52:49.487 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.783+0000 7fee66a80640 1 -- 172.21.15.103:0/871365180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee601082d0 con 0x7fee60107e70 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.785+0000 7fee5dffb640 1 -- 172.21.15.103:0/871365180 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fee5403b980 con 0x7fee60107e70 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.846+0000 7fee66a80640 1 -- 172.21.15.103:0/871365180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fee6019ed70 con 0x7fee60107e70 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.846+0000 7fee5dffb640 1 -- 172.21.15.103:0/871365180 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fee54044d50 con 0x7fee60107e70 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.847+0000 7fee66a80640 1 -- 172.21.15.103:0/871365180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 msgr2=0x7fee6019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.847+0000 7fee66a80640 1 --2- 172.21.15.103:0/871365180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6019b330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fee540097f0 tx=0x7fee54030d00 comp rx=0 tx=0).stop 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.847+0000 7fee66a80640 1 -- 172.21.15.103:0/871365180 shutdown_connections 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.847+0000 7fee66a80640 1 --2- 172.21.15.103:0/871365180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee60107e70 0x7fee6019b330 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.847+0000 7fee66a80640 1 -- 172.21.15.103:0/871365180 >> 172.21.15.103:0/871365180 conn(0x7fee60101a50 msgr2=0x7fee60102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.847+0000 7fee66a80640 1 -- 172.21.15.103:0/871365180 shutdown_connections 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:48.847+0000 7fee66a80640 1 -- 172.21.15.103:0/871365180 wait complete. 2024-09-05T01:52:49.488 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (1/15)... 2024-09-05T01:52:49.898 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:49 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:49.509+0000 7f299cf52040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:52:50.648 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:50 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:50.389+0000 7f299cf52040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:52:50.648 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:50 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:50.456+0000 7f299cf52040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:52:51.432 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:51.175+0000 7f299cf52040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:52:51.433 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:51.289+0000 7f299cf52040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:52:51.433 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:51.432+0000 7f299cf52040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:52:51.703 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:51.501+0000 7f299cf52040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:52:51.703 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:51.654+0000 7f299cf52040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:52:51.971 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:51.726+0000 7f299cf52040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:52:51.971 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:51.787+0000 7f299cf52040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:52:51.971 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:51.970+0000 7f299cf52040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:52:52.253 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:52 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1749272988' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T01:52:52.253 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:52.041+0000 7f299cf52040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:52:52.253 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:52.109+0000 7f299cf52040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:52:52.253 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:52.191+0000 7f299cf52040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:52:52.253 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:52.252+0000 7f299cf52040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:52:52.613 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:52.313+0000 7f299cf52040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:52:52.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:52.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:52:52.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:52:52.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-09-05T01:52:52.752 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T01:52:52.752 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:52.752 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T01:52:52.752 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T01:52:52.752 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:52.752 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T01:52:52.752 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:52.752 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T01:52:52.752 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T01:52:52.760 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T01:52:52.760 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T01:52:52.760 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T01:52:52.760 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T01:52:52.760 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:52.760 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T01:52:29:043577+0000", 2024-09-05T01:52:52.761 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T01:52:52.762 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T01:52:29.046314+0000", 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.890+0000 7f26ef915640 1 Processor -- start 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.890+0000 7f26ef915640 1 -- start start 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.891+0000 7f26ef915640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e8106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.891+0000 7f26ef915640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8106cd0 con 0x7f26e8106300 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.891+0000 7f26ed6b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e8106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.891+0000 7f26ed6b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e8106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57344/0 (socket says 172.21.15.103:57344) 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.891+0000 7f26ed6b1640 1 -- 172.21.15.103:0/2424533893 learned_addr learned my addr 172.21.15.103:0/2424533893 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:52.763 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.891+0000 7f26ed6b1640 1 -- 172.21.15.103:0/2424533893 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e8107500 con 0x7f26e8106300 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.892+0000 7f26ed6b1640 1 --2- 172.21.15.103:0/2424533893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e8106700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009820 tx=0x7f26e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c2a8347e5b46cc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.892+0000 7f26dffff640 1 -- 172.21.15.103:0/2424533893 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26e4037070 con 0x7f26e8106300 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.892+0000 7f26dffff640 1 -- 172.21.15.103:0/2424533893 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f26e40328d0 con 0x7f26e8106300 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.892+0000 7f26dffff640 1 -- 172.21.15.103:0/2424533893 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26e40307d0 con 0x7f26e8106300 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.893+0000 7f26ef915640 1 -- 172.21.15.103:0/2424533893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 msgr2=0x7f26e8106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.893+0000 7f26ef915640 1 --2- 172.21.15.103:0/2424533893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e8106700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009820 tx=0x7f26e402a270 comp rx=0 tx=0).stop 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.893+0000 7f26ef915640 1 -- 172.21.15.103:0/2424533893 shutdown_connections 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.893+0000 7f26ef915640 1 --2- 172.21.15.103:0/2424533893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e8106700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.893+0000 7f26ef915640 1 -- 172.21.15.103:0/2424533893 >> 172.21.15.103:0/2424533893 conn(0x7f26e8101a50 msgr2=0x7f26e8103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.893+0000 7f26ef915640 1 -- 172.21.15.103:0/2424533893 shutdown_connections 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.893+0000 7f26ef915640 1 -- 172.21.15.103:0/2424533893 wait complete. 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.894+0000 7f26ef915640 1 Processor -- start 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.894+0000 7f26ef915640 1 -- start start 2024-09-05T01:52:52.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.894+0000 7f26ef915640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.894+0000 7f26ef915640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26e810a8b0 con 0x7f26e8106300 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.894+0000 7f26ed6b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.894+0000 7f26ed6b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57356/0 (socket says 172.21.15.103:57356) 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.894+0000 7f26ed6b1640 1 -- 172.21.15.103:0/1749272988 learned_addr learned my addr 172.21.15.103:0/1749272988 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.895+0000 7f26ed6b1640 1 -- 172.21.15.103:0/1749272988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e819a780 con 0x7f26e8106300 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.895+0000 7f26ed6b1640 1 --2- 172.21.15.103:0/1749272988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e819a240 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26e4030520 tx=0x7f26e4032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.895+0000 7f26de7fc640 1 -- 172.21.15.103:0/1749272988 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26e4042070 con 0x7f26e8106300 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.895+0000 7f26de7fc640 1 -- 172.21.15.103:0/1749272988 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f26e4030cc0 con 0x7f26e8106300 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.895+0000 7f26ef915640 1 -- 172.21.15.103:0/1749272988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e819aa10 con 0x7f26e8106300 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.895+0000 7f26de7fc640 1 -- 172.21.15.103:0/1749272988 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26e4037040 con 0x7f26e8106300 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.895+0000 7f26ef915640 1 -- 172.21.15.103:0/1749272988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e819d700 con 0x7f26e8106300 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.895+0000 7f26de7fc640 1 -- 172.21.15.103:0/1749272988 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f26e4045070 con 0x7f26e8106300 2024-09-05T01:52:52.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.895+0000 7f26de7fc640 1 -- 172.21.15.103:0/1749272988 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f26e40479a0 con 0x7f26e8106300 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.896+0000 7f26ef915640 1 -- 172.21.15.103:0/1749272988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e810aa30 con 0x7f26e8106300 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.898+0000 7f26de7fc640 1 -- 172.21.15.103:0/1749272988 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f26e403d9c0 con 0x7f26e8106300 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.980+0000 7f26ef915640 1 -- 172.21.15.103:0/1749272988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f26e819d8d0 con 0x7f26e8106300 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.981+0000 7f26de7fc640 1 -- 172.21.15.103:0/1749272988 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f26e403d430 con 0x7f26e8106300 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.982+0000 7f26ef915640 1 -- 172.21.15.103:0/1749272988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 msgr2=0x7f26e819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.982+0000 7f26ef915640 1 --2- 172.21.15.103:0/1749272988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e819a240 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26e4030520 tx=0x7f26e4032dd0 comp rx=0 tx=0).stop 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.982+0000 7f26ef915640 1 -- 172.21.15.103:0/1749272988 shutdown_connections 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.982+0000 7f26ef915640 1 --2- 172.21.15.103:0/1749272988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26e8106300 0x7f26e819a240 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.982+0000 7f26ef915640 1 -- 172.21.15.103:0/1749272988 >> 172.21.15.103:0/1749272988 conn(0x7f26e8101a50 msgr2=0x7f26e8102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.982+0000 7f26ef915640 1 -- 172.21.15.103:0/1749272988 shutdown_connections 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:51.982+0000 7f26ef915640 1 -- 172.21.15.103:0/1749272988 wait complete. 2024-09-05T01:52:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (2/15)... 2024-09-05T01:52:53.398 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:53.110+0000 7f299cf52040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:52:53.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: Activating manager daemon x 2024-09-05T01:52:53.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: mgrmap e2: x(active, starting, since 0.00300036s) 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: Manager daemon x is now available 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' 2024-09-05T01:52:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:53 smithi103 ceph-mon[30902]: from='mgr.14100 172.21.15.103:0/1510974279' entity='mgr.x' 2024-09-05T01:52:55.118 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:55 smithi103 ceph-mon[30902]: mgrmap e3: x(active, since 1.00757s) 2024-09-05T01:52:55.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:55.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:52:55.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T01:52:55.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T01:52:55.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T01:52:55.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T01:52:55.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T01:52:55.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:55.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T01:52:55.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:55.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T01:52:55.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T01:52:55.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T01:52:55.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T01:52:55.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T01:52:55.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T01:52:55.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:55.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T01:52:55.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T01:52:55.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T01:52:55.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T01:52:55.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T01:52:29:043577+0000", 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T01:52:55.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T01:52:29.046314+0000", 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:52:55.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.111+0000 7f8f70e5a640 1 Processor -- start 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.111+0000 7f8f70e5a640 1 -- start start 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.111+0000 7f8f70e5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.111+0000 7f8f70e5a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c106cd0 con 0x7f8f6c106300 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.111+0000 7f8f6a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.111+0000 7f8f6a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57418/0 (socket says 172.21.15.103:57418) 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.111+0000 7f8f6a59c640 1 -- 172.21.15.103:0/1214078760 learned_addr learned my addr 172.21.15.103:0/1214078760 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.111+0000 7f8f6a59c640 1 -- 172.21.15.103:0/1214078760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c107500 con 0x7f8f6c106300 2024-09-05T01:52:55.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.112+0000 7f8f6a59c640 1 --2- 172.21.15.103:0/1214078760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c106700 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009820 tx=0x7f8f5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67882b819ba18a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.112+0000 7f8f6959a640 1 -- 172.21.15.103:0/1214078760 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f5c037070 con 0x7f8f6c106300 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.112+0000 7f8f6959a640 1 -- 172.21.15.103:0/1214078760 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8f5c0328d0 con 0x7f8f6c106300 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.112+0000 7f8f6959a640 1 -- 172.21.15.103:0/1214078760 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f5c0307d0 con 0x7f8f6c106300 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.113+0000 7f8f70e5a640 1 -- 172.21.15.103:0/1214078760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 msgr2=0x7f8f6c106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.113+0000 7f8f70e5a640 1 --2- 172.21.15.103:0/1214078760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c106700 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009820 tx=0x7f8f5c02a270 comp rx=0 tx=0).stop 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.113+0000 7f8f70e5a640 1 -- 172.21.15.103:0/1214078760 shutdown_connections 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.113+0000 7f8f70e5a640 1 --2- 172.21.15.103:0/1214078760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c106700 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.113+0000 7f8f70e5a640 1 -- 172.21.15.103:0/1214078760 >> 172.21.15.103:0/1214078760 conn(0x7f8f6c101ad0 msgr2=0x7f8f6c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.113+0000 7f8f70e5a640 1 -- 172.21.15.103:0/1214078760 shutdown_connections 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.113+0000 7f8f70e5a640 1 -- 172.21.15.103:0/1214078760 wait complete. 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.114+0000 7f8f70e5a640 1 Processor -- start 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.114+0000 7f8f70e5a640 1 -- start start 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.114+0000 7f8f70e5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:55.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.114+0000 7f8f70e5a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c10a8b0 con 0x7f8f6c106300 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.115+0000 7f8f6a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.115+0000 7f8f6a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57434/0 (socket says 172.21.15.103:57434) 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.115+0000 7f8f6a59c640 1 -- 172.21.15.103:0/3236796221 learned_addr learned my addr 172.21.15.103:0/3236796221 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.115+0000 7f8f6a59c640 1 -- 172.21.15.103:0/3236796221 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c19a780 con 0x7f8f6c106300 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.115+0000 7f8f6a59c640 1 --2- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c19a240 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c030520 tx=0x7f8f5c032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.117+0000 7f8f5b7fe640 1 -- 172.21.15.103:0/3236796221 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f5c042070 con 0x7f8f6c106300 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.117+0000 7f8f5b7fe640 1 -- 172.21.15.103:0/3236796221 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8f5c030cc0 con 0x7f8f6c106300 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.117+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c19aa10 con 0x7f8f6c106300 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.117+0000 7f8f5b7fe640 1 -- 172.21.15.103:0/3236796221 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f5c037040 con 0x7f8f6c106300 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.117+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c19d700 con 0x7f8f6c106300 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.118+0000 7f8f5b7fe640 1 -- 172.21.15.103:0/3236796221 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f8f5c045070 con 0x7f8f6c106300 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.118+0000 7f8f5b7fe640 1 --2- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f8f54038ed0 0x7f8f5403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:55.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.118+0000 7f8f5b7fe640 1 -- 172.21.15.103:0/3236796221 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8f5c06bd50 con 0x7f8f6c106300 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.118+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c10aa30 con 0x7f8f6c106300 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.118+0000 7f8f69d9b640 1 --2- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f8f54038ed0 0x7f8f5403b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.119+0000 7f8f69d9b640 1 --2- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f8f54038ed0 0x7f8f5403b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8f600098c0 tx=0x7f8f60006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.119+0000 7f8f5b7fe640 1 -- 172.21.15.103:0/3236796221 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8f5c03cb00 con 0x7f8f6c106300 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.126+0000 7f8f5b7fe640 1 -- 172.21.15.103:0/3236796221 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f5c046e00 con 0x7f8f6c106300 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.355+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8f6c07af70 con 0x7f8f6c106300 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.356+0000 7f8f5b7fe640 1 -- 172.21.15.103:0/3236796221 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f8f5c045360 con 0x7f8f6c106300 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f8f54038ed0 msgr2=0x7f8f5403b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 --2- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f8f54038ed0 0x7f8f5403b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8f600098c0 tx=0x7f8f60006db0 comp rx=0 tx=0).stop 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 msgr2=0x7f8f6c19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 --2- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c19a240 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c030520 tx=0x7f8f5c032dd0 comp rx=0 tx=0).stop 2024-09-05T01:52:55.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 shutdown_connections 2024-09-05T01:52:55.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 --2- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f8f54038ed0 0x7f8f5403b390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:55.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 --2- 172.21.15.103:0/3236796221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f6c106300 0x7f8f6c19a240 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:55.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 >> 172.21.15.103:0/3236796221 conn(0x7f8f6c101ad0 msgr2=0x7f8f6c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:55.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 shutdown_connections 2024-09-05T01:52:55.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:55.358+0000 7f8f70e5a640 1 -- 172.21.15.103:0/3236796221 wait complete. 2024-09-05T01:52:55.929 INFO:teuthology.orchestra.run.smithi103.stdout:mgr is available 2024-09-05T01:52:56.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:56 smithi103 ceph-mon[30902]: mgrmap e4: x(active, since 2s) 2024-09-05T01:52:56.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:56 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3236796221' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T01:52:56.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:56.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T01:52:56.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:52:56.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-05T01:52:56.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T01:52:56.991 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.309+0000 7f81fbaec640 1 Processor -- start 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.309+0000 7f81fbaec640 1 -- start start 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.309+0000 7f81fbaec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.309+0000 7f81fbaec640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4106cd0 con 0x7f81f4106300 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.310+0000 7f81f9888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.310+0000 7f81f9888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57450/0 (socket says 172.21.15.103:57450) 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.310+0000 7f81f9888640 1 -- 172.21.15.103:0/3984124027 learned_addr learned my addr 172.21.15.103:0/3984124027 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.310+0000 7f81f9888640 1 -- 172.21.15.103:0/3984124027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f4107500 con 0x7f81f4106300 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.311+0000 7f81f9888640 1 --2- 172.21.15.103:0/3984124027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4106700 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009820 tx=0x7f81e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59dad58df48ac7d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.311+0000 7f81f8886640 1 -- 172.21.15.103:0/3984124027 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81e4037070 con 0x7f81f4106300 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.311+0000 7f81f8886640 1 -- 172.21.15.103:0/3984124027 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f81e40328d0 con 0x7f81f4106300 2024-09-05T01:52:56.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.311+0000 7f81f8886640 1 -- 172.21.15.103:0/3984124027 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81e40307d0 con 0x7f81f4106300 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.311+0000 7f81fbaec640 1 -- 172.21.15.103:0/3984124027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 msgr2=0x7f81f4106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.311+0000 7f81fbaec640 1 --2- 172.21.15.103:0/3984124027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4106700 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009820 tx=0x7f81e402a270 comp rx=0 tx=0).stop 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.311+0000 7f81fbaec640 1 -- 172.21.15.103:0/3984124027 shutdown_connections 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.311+0000 7f81fbaec640 1 --2- 172.21.15.103:0/3984124027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4106700 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.311+0000 7f81fbaec640 1 -- 172.21.15.103:0/3984124027 >> 172.21.15.103:0/3984124027 conn(0x7f81f4101a50 msgr2=0x7f81f4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.312+0000 7f81fbaec640 1 -- 172.21.15.103:0/3984124027 shutdown_connections 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.312+0000 7f81fbaec640 1 -- 172.21.15.103:0/3984124027 wait complete. 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.312+0000 7f81fbaec640 1 Processor -- start 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.312+0000 7f81fbaec640 1 -- start start 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.312+0000 7f81fbaec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.312+0000 7f81fbaec640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81f410a8b0 con 0x7f81f4106300 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.312+0000 7f81f9888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.312+0000 7f81f9888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4075340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57460/0 (socket says 172.21.15.103:57460) 2024-09-05T01:52:56.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.312+0000 7f81f9888640 1 -- 172.21.15.103:0/1615398249 learned_addr learned my addr 172.21.15.103:0/1615398249 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.313+0000 7f81f9888640 1 -- 172.21.15.103:0/1615398249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f419e090 con 0x7f81f4106300 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.313+0000 7f81f9888640 1 --2- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4075340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f81e4030520 tx=0x7f81e4032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.313+0000 7f81f2ffd640 1 -- 172.21.15.103:0/1615398249 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81e4042070 con 0x7f81f4106300 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.313+0000 7f81f2ffd640 1 -- 172.21.15.103:0/1615398249 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f81e4030cc0 con 0x7f81f4106300 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.313+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f419e260 con 0x7f81f4106300 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.313+0000 7f81f2ffd640 1 -- 172.21.15.103:0/1615398249 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81e4037040 con 0x7f81f4106300 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.313+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f419e600 con 0x7f81f4106300 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.314+0000 7f81f2ffd640 1 -- 172.21.15.103:0/1615398249 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f81e4045070 con 0x7f81f4106300 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.314+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f410aa30 con 0x7f81f4106300 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.314+0000 7f81f2ffd640 1 --2- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f81e0039030 0x7f81e003b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.314+0000 7f81f2ffd640 1 -- 172.21.15.103:0/1615398249 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f81e406bfe0 con 0x7f81f4106300 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.314+0000 7f81f9087640 1 --2- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f81e0039030 0x7f81e003b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:56.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.315+0000 7f81f9087640 1 --2- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f81e0039030 0x7f81e003b4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00ad30 tx=0x7f81dc0093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.319+0000 7f81f2ffd640 1 -- 172.21.15.103:0/1615398249 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81e4046dd0 con 0x7f81f4106300 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.487+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f81f4106780 con 0x7f81f4106300 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.488+0000 7f81f2ffd640 1 -- 172.21.15.103:0/1615398249 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f81e4050df0 con 0x7f81f4106300 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f81e0039030 msgr2=0x7f81e003b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 --2- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f81e0039030 0x7f81e003b4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00ad30 tx=0x7f81dc0093f0 comp rx=0 tx=0).stop 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 msgr2=0x7f81f4075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 --2- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4075340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f81e4030520 tx=0x7f81e4032dd0 comp rx=0 tx=0).stop 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 shutdown_connections 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 --2- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f81e0039030 0x7f81e003b4f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 --2- 172.21.15.103:0/1615398249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81f4106300 0x7f81f4075340 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 >> 172.21.15.103:0/1615398249 conn(0x7f81f4101a50 msgr2=0x7f81f4102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 shutdown_connections 2024-09-05T01:52:56.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:56.490+0000 7f81fbaec640 1 -- 172.21.15.103:0/1615398249 wait complete. 2024-09-05T01:52:56.996 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling cephadm module... 2024-09-05T01:52:57.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:57 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1615398249' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-05T01:52:58.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:58 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/839461885' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-05T01:52:58.398 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: ignoring --setuser ceph since I am not root 2024-09-05T01:52:58.399 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: ignoring --setgroup ceph since I am not root 2024-09-05T01:52:58.399 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:58.262+0000 7fe9ed315040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:52:58.399 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:58.319+0000 7fe9ed315040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:52:58.687 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:58.469+0000 7fe9ed315040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:52:58.687 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:58.536+0000 7fe9ed315040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:52:58.688 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:58.686+0000 7fe9ed315040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:52:58.721 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.326+0000 7f770e509640 1 Processor -- start 2024-09-05T01:52:58.721 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.326+0000 7f770e509640 1 -- start start 2024-09-05T01:52:58.721 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.326+0000 7f770e509640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f7708107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:58.721 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.326+0000 7f770e509640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7708107e10 con 0x7f7708107440 2024-09-05T01:52:58.721 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.326+0000 7f7707fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f7708107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:58.721 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.326+0000 7f7707fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f7708107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57462/0 (socket says 172.21.15.103:57462) 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.326+0000 7f7707fff640 1 -- 172.21.15.103:0/1685321137 learned_addr learned my addr 172.21.15.103:0/1685321137 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.326+0000 7f7707fff640 1 -- 172.21.15.103:0/1685321137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7708108640 con 0x7f7708107440 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.327+0000 7f7707fff640 1 --2- 172.21.15.103:0/1685321137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f7708107840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009820 tx=0x7f76f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af754b66bd80004b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.327+0000 7f7706ffd640 1 -- 172.21.15.103:0/1685321137 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76f8037070 con 0x7f7708107440 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.327+0000 7f7706ffd640 1 -- 172.21.15.103:0/1685321137 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f76f80328d0 con 0x7f7708107440 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.327+0000 7f7706ffd640 1 -- 172.21.15.103:0/1685321137 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76f80307d0 con 0x7f7708107440 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.328+0000 7f770e509640 1 -- 172.21.15.103:0/1685321137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 msgr2=0x7f7708107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.328+0000 7f770e509640 1 --2- 172.21.15.103:0/1685321137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f7708107840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009820 tx=0x7f76f802a270 comp rx=0 tx=0).stop 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.328+0000 7f770e509640 1 -- 172.21.15.103:0/1685321137 shutdown_connections 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.328+0000 7f770e509640 1 --2- 172.21.15.103:0/1685321137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f7708107840 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.328+0000 7f770e509640 1 -- 172.21.15.103:0/1685321137 >> 172.21.15.103:0/1685321137 conn(0x7f7708102bb0 msgr2=0x7f7708105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.328+0000 7f770e509640 1 -- 172.21.15.103:0/1685321137 shutdown_connections 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.328+0000 7f770e509640 1 -- 172.21.15.103:0/1685321137 wait complete. 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.328+0000 7f770e509640 1 Processor -- start 2024-09-05T01:52:58.722 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f770e509640 1 -- start start 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f770e509640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f770819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f770e509640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f770810b9f0 con 0x7f7708107440 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f7707fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f770819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f7707fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f770819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57474/0 (socket says 172.21.15.103:57474) 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f7707fff640 1 -- 172.21.15.103:0/839461885 learned_addr learned my addr 172.21.15.103:0/839461885 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f7707fff640 1 -- 172.21.15.103:0/839461885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770819b820 con 0x7f7708107440 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f7707fff640 1 --2- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f770819b2e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f76f80097f0 tx=0x7f76f8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f77057fa640 1 -- 172.21.15.103:0/839461885 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76f8037040 con 0x7f7708107440 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770819bab0 con 0x7f7708107440 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f77057fa640 1 -- 172.21.15.103:0/839461885 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f76f8032bb0 con 0x7f7708107440 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f77057fa640 1 -- 172.21.15.103:0/839461885 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76f803a390 con 0x7f7708107440 2024-09-05T01:52:58.723 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.329+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770819e7a0 con 0x7f7708107440 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.330+0000 7f77057fa640 1 -- 172.21.15.103:0/839461885 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f76f803a5d0 con 0x7f7708107440 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.330+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770810bb70 con 0x7f7708107440 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.330+0000 7f77057fa640 1 --2- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f76f0038fe0 0x7f76f003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.331+0000 7f77057fa640 1 -- 172.21.15.103:0/839461885 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f76f806c230 con 0x7f7708107440 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.331+0000 7f77077fe640 1 --2- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f76f0038fe0 0x7f76f003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.331+0000 7f77077fe640 1 --2- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f76f0038fe0 0x7f76f003b4a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0098c0 tx=0x7f76fc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.335+0000 7f77057fa640 1 -- 172.21.15.103:0/839461885 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76f80313a0 con 0x7f7708107440 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:57.545+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f770819ee10 con 0x7f7708107440 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.122+0000 7f77057fa640 1 -- 172.21.15.103:0/839461885 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f76f803a970 con 0x7f7708107440 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.123+0000 7f77057fa640 1 -- 172.21.15.103:0/839461885 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f76f803ac40 con 0x7f7708107440 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f76f0038fe0 msgr2=0x7f76f003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 --2- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f76f0038fe0 0x7f76f003b4a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0098c0 tx=0x7f76fc006db0 comp rx=0 tx=0).stop 2024-09-05T01:52:58.724 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 msgr2=0x7f770819b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:58.725 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 --2- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f770819b2e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f76f80097f0 tx=0x7f76f8030d00 comp rx=0 tx=0).stop 2024-09-05T01:52:58.725 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 shutdown_connections 2024-09-05T01:52:58.725 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 --2- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f76f0038fe0 0x7f76f003b4a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:58.725 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 --2- 172.21.15.103:0/839461885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7708107440 0x7f770819b2e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:58.725 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 >> 172.21.15.103:0/839461885 conn(0x7f7708102bb0 msgr2=0x7f77081035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:58.725 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 shutdown_connections 2024-09-05T01:52:58.725 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:58.125+0000 7f770e509640 1 -- 172.21.15.103:0/839461885 wait complete. 2024-09-05T01:52:59.015 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:58.832+0000 7fe9ed315040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:52:59.299 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:59 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/839461885' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-05T01:52:59.299 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:52:59 smithi103 ceph-mon[30902]: mgrmap e5: x(active, since 5s) 2024-09-05T01:52:59.300 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:59.157+0000 7fe9ed315040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:52:59.300 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:59.219+0000 7fe9ed315040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:52:59.648 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:52:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:52:59.412+0000 7fe9ed315040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:52:59.945 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:52:59.945 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-05T01:52:59.945 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.091+0000 7f6b8c034640 1 Processor -- start 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.091+0000 7f6b8c034640 1 -- start start 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.091+0000 7f6b8c034640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8410a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.091+0000 7f6b8c034640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84075760 con 0x7f6b84107eb0 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.091+0000 7f6b89dd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8410a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.092+0000 7f6b89dd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8410a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57514/0 (socket says 172.21.15.103:57514) 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.092+0000 7f6b89dd0640 1 -- 172.21.15.103:0/2231524081 learned_addr learned my addr 172.21.15.103:0/2231524081 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.092+0000 7f6b89dd0640 1 -- 172.21.15.103:0/2231524081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8410a7e0 con 0x7f6b84107eb0 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.093+0000 7f6b89dd0640 1 --2- 172.21.15.103:0/2231524081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8410a2a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009820 tx=0x7f6b8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f321b3ce884727 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:59.946 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.093+0000 7f6b88dce640 1 -- 172.21.15.103:0/2231524081 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b80037070 con 0x7f6b84107eb0 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.093+0000 7f6b88dce640 1 -- 172.21.15.103:0/2231524081 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6b800328d0 con 0x7f6b84107eb0 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.093+0000 7f6b88dce640 1 -- 172.21.15.103:0/2231524081 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b800307d0 con 0x7f6b84107eb0 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.094+0000 7f6b8c034640 1 -- 172.21.15.103:0/2231524081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 msgr2=0x7f6b8410a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.094+0000 7f6b8c034640 1 --2- 172.21.15.103:0/2231524081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8410a2a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009820 tx=0x7f6b8002a270 comp rx=0 tx=0).stop 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.094+0000 7f6b8c034640 1 -- 172.21.15.103:0/2231524081 shutdown_connections 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.094+0000 7f6b8c034640 1 --2- 172.21.15.103:0/2231524081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8410a2a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.094+0000 7f6b8c034640 1 -- 172.21.15.103:0/2231524081 >> 172.21.15.103:0/2231524081 conn(0x7f6b84101a50 msgr2=0x7f6b84103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.094+0000 7f6b8c034640 1 -- 172.21.15.103:0/2231524081 shutdown_connections 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.094+0000 7f6b8c034640 1 -- 172.21.15.103:0/2231524081 wait complete. 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.095+0000 7f6b8c034640 1 Processor -- start 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.095+0000 7f6b8c034640 1 -- start start 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.095+0000 7f6b8c034640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:59.947 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.095+0000 7f6b8c034640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8410b7c0 con 0x7f6b84107eb0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.095+0000 7f6b89dd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.095+0000 7f6b89dd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57520/0 (socket says 172.21.15.103:57520) 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.095+0000 7f6b89dd0640 1 -- 172.21.15.103:0/1356148625 learned_addr learned my addr 172.21.15.103:0/1356148625 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.095+0000 7f6b89dd0640 1 -- 172.21.15.103:0/1356148625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8419b8a0 con 0x7f6b84107eb0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.095+0000 7f6b89dd0640 1 --2- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8419b360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6b80030520 tx=0x7f6b80032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.096+0000 7f6b7affd640 1 -- 172.21.15.103:0/1356148625 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b80042070 con 0x7f6b84107eb0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.096+0000 7f6b7affd640 1 -- 172.21.15.103:0/1356148625 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6b80030cc0 con 0x7f6b84107eb0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.096+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8419bb30 con 0x7f6b84107eb0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.096+0000 7f6b7affd640 1 -- 172.21.15.103:0/1356148625 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b80037040 con 0x7f6b84107eb0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.096+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8419e820 con 0x7f6b84107eb0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.097+0000 7f6b7affd640 1 -- 172.21.15.103:0/1356148625 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f6b80045070 con 0x7f6b84107eb0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.097+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b84108310 con 0x7f6b84107eb0 2024-09-05T01:52:59.948 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.097+0000 7f6b7affd640 1 --2- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f6b70039080 0x7f6b7003b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.097+0000 7f6b7affd640 1 -- 172.21.15.103:0/1356148625 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6b8006bf30 con 0x7f6b84107eb0 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.097+0000 7f6b895cf640 1 -- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f6b70039080 msgr2=0x7f6b7003b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2132077607 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.097+0000 7f6b895cf640 1 --2- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f6b70039080 0x7f6b7003b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.102+0000 7f6b7affd640 1 -- 172.21.15.103:0/1356148625 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b80045360 con 0x7f6b84107eb0 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.298+0000 7f6b895cf640 1 -- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f6b70039080 msgr2=0x7f6b7003b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2132077607 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.298+0000 7f6b895cf640 1 --2- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f6b70039080 0x7f6b7003b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.322+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6b8419eea0 con 0x7f6b84107eb0 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.322+0000 7f6b7affd640 1 -- 172.21.15.103:0/1356148625 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6b8006b950 con 0x7f6b84107eb0 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f6b70039080 msgr2=0x7f6b7003b540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 --2- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f6b70039080 0x7f6b7003b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 msgr2=0x7f6b8419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:52:59.949 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 --2- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8419b360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6b80030520 tx=0x7f6b80032dd0 comp rx=0 tx=0).stop 2024-09-05T01:52:59.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 shutdown_connections 2024-09-05T01:52:59.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 --2- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f6b70039080 0x7f6b7003b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:59.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 --2- 172.21.15.103:0/1356148625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b84107eb0 0x7f6b8419b360 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:52:59.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 >> 172.21.15.103:0/1356148625 conn(0x7f6b84101a50 msgr2=0x7f6b84102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:52:59.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 shutdown_connections 2024-09-05T01:52:59.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:52:59.325+0000 7f6b8c034640 1 -- 172.21.15.103:0/1356148625 wait complete. 2024-09-05T01:52:59.950 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-05T01:52:59.950 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 5... 2024-09-05T01:53:00.215 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:00 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1356148625' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T01:53:00.501 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:00 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:00.319+0000 7fe9ed315040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:53:00.501 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:00 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:00.389+0000 7fe9ed315040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:53:01.250 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:01 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:01.129+0000 7fe9ed315040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:53:01.250 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:01 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:01.248+0000 7fe9ed315040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:53:01.605 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:01 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:01.396+0000 7fe9ed315040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:53:01.605 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:01 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:01.457+0000 7fe9ed315040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:53:01.605 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:01 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:01.603+0000 7fe9ed315040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:53:01.890 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:01 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:01.671+0000 7fe9ed315040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:53:01.890 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:01 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:01.732+0000 7fe9ed315040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:53:01.890 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:01 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:01.889+0000 7fe9ed315040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:53:02.148 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:01 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:01.954+0000 7fe9ed315040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:53:02.148 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:02 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:02.020+0000 7fe9ed315040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:53:02.148 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:02 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:02.100+0000 7fe9ed315040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:53:02.648 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:02 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:02.159+0000 7fe9ed315040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:53:02.648 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:02 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:02.218+0000 7fe9ed315040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:53:03.398 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:02 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:02.997+0000 7fe9ed315040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:53:03.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: Active manager daemon x restarted 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: Activating manager daemon x 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: osdmap e2: 0 total, 0 up, 0 in 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: mgrmap e6: x(active, starting, since 0.00397003s) 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: Manager daemon x is now available 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:03.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T01:53:03.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:03 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T01:53:04.325 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-mon[30902]: Found migration_current of "None". Setting to last migration. 2024-09-05T01:53:04.326 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:04.326 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:04.326 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-mon[30902]: mgrmap e7: x(active, since 1.00546s) 2024-09-05T01:53:04.585 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.293+0000 7f9cb8bdc640 1 Processor -- start 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.293+0000 7f9cb8bdc640 1 -- start start 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.293+0000 7f9cb8bdc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb410a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.293+0000 7f9cb8bdc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb40756c0 con 0x7f9cb4107f60 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.293+0000 7f9cb259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb410a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.293+0000 7f9cb259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb410a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40670/0 (socket says 172.21.15.103:40670) 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.293+0000 7f9cb259c640 1 -- 172.21.15.103:0/928206946 learned_addr learned my addr 172.21.15.103:0/928206946 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:04.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.293+0000 7f9cb259c640 1 -- 172.21.15.103:0/928206946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb410a890 con 0x7f9cb4107f60 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.294+0000 7f9cb259c640 1 --2- 172.21.15.103:0/928206946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb410a350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009820 tx=0x7f9ca402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41695765d0641cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.294+0000 7f9cb159a640 1 -- 172.21.15.103:0/928206946 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4037070 con 0x7f9cb4107f60 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.294+0000 7f9cb159a640 1 -- 172.21.15.103:0/928206946 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9ca40328d0 con 0x7f9cb4107f60 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.294+0000 7f9cb159a640 1 -- 172.21.15.103:0/928206946 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ca40307d0 con 0x7f9cb4107f60 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.295+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/928206946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 msgr2=0x7f9cb410a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.295+0000 7f9cb8bdc640 1 --2- 172.21.15.103:0/928206946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb410a350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009820 tx=0x7f9ca402a270 comp rx=0 tx=0).stop 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.295+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/928206946 shutdown_connections 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.295+0000 7f9cb8bdc640 1 --2- 172.21.15.103:0/928206946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb410a350 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.295+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/928206946 >> 172.21.15.103:0/928206946 conn(0x7f9cb4101a60 msgr2=0x7f9cb4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.295+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/928206946 shutdown_connections 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.295+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/928206946 wait complete. 2024-09-05T01:53:04.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.296+0000 7f9cb8bdc640 1 Processor -- start 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.296+0000 7f9cb8bdc640 1 -- start start 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.297+0000 7f9cb8bdc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb41077e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.297+0000 7f9cb8bdc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb410b7b0 con 0x7f9cb4107f60 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.297+0000 7f9cb259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb41077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.297+0000 7f9cb259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb41077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40678/0 (socket says 172.21.15.103:40678) 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.297+0000 7f9cb259c640 1 -- 172.21.15.103:0/2018515638 learned_addr learned my addr 172.21.15.103:0/2018515638 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.297+0000 7f9cb259c640 1 -- 172.21.15.103:0/2018515638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb4107d20 con 0x7f9cb4107f60 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.297+0000 7f9cb259c640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb41077e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40097f0 tx=0x7f9ca4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.297+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4037040 con 0x7f9cb4107f60 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.297+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb4105f10 con 0x7f9cb4107f60 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.298+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9ca403b3f0 con 0x7f9cb4107f60 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.298+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ca403a520 con 0x7f9cb4107f60 2024-09-05T01:53:04.588 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.298+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb4106450 con 0x7f9cb4107f60 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.298+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f9ca403a710 con 0x7f9cb4107f60 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.299+0000 7f9ca37fe640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 0x7f9c9c03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.299+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9ca403b980 con 0x7f9cb4107f60 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.299+0000 7f9cb1d9b640 1 -- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 msgr2=0x7f9c9c03b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2132077607 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.299+0000 7f9cb1d9b640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 0x7f9c9c03b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.299+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 --> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9cb41083c0 con 0x7f9c9c039030 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.500+0000 7f9cb1d9b640 1 -- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 msgr2=0x7f9c9c03b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2132077607 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.500+0000 7f9cb1d9b640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 0x7f9c9c03b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.901+0000 7f9cb1d9b640 1 -- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 msgr2=0x7f9c9c03b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2132077607 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:00.901+0000 7f9cb1d9b640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 0x7f9c9c03b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:01.702+0000 7f9cb1d9b640 1 -- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 msgr2=0x7f9c9c03b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2132077607 2024-09-05T01:53:04.589 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:01.702+0000 7f9cb1d9b640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 0x7f9c9c03b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:03.001+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f9ca4044920 con 0x7f9cb4107f60 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:03.001+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 msgr2=0x7f9c9c03b4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:03.001+0000 7f9ca37fe640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 0x7f9c9c03b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.002+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f9ca4044400 con 0x7f9cb4107f60 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.002+0000 7f9ca37fe640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9c9c03c770 0x7f9c9c03eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.002+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9ca403de50 con 0x7f9c9c03c770 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.003+0000 7f9cb1d9b640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9c9c03c770 0x7f9c9c03eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.003+0000 7f9cb1d9b640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9c9c03c770 0x7f9c9c03eb60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8003980 tx=0x7f9ca8006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.004+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 <== mgr.14118 v2:172.21.15.103:6800/2384335573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f9ca403de50 con 0x7f9c9c03c770 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.016+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9cb4102d50 con 0x7f9c9c03c770 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.016+0000 7f9ca37fe640 1 -- 172.21.15.103:0/2018515638 <== mgr.14118 v2:172.21.15.103:6800/2384335573 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f9cb4102d50 con 0x7f9c9c03c770 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.016+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9c9c03c770 msgr2=0x7f9c9c03eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.016+0000 7f9cb8bdc640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9c9c03c770 0x7f9c9c03eb60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8003980 tx=0x7f9ca8006e10 comp rx=0 tx=0).stop 2024-09-05T01:53:04.590 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.016+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 msgr2=0x7f9cb41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:04.591 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.016+0000 7f9cb8bdc640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb41077e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40097f0 tx=0x7f9ca4030d00 comp rx=0 tx=0).stop 2024-09-05T01:53:04.591 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.017+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 shutdown_connections 2024-09-05T01:53:04.591 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.017+0000 7f9cb8bdc640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2132077607,v1:172.21.15.103:6801/2132077607] conn(0x7f9c9c039030 0x7f9c9c03b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:04.591 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.017+0000 7f9cb8bdc640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9c9c03c770 0x7f9c9c03eb60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:04.591 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.017+0000 7f9cb8bdc640 1 --2- 172.21.15.103:0/2018515638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cb4107f60 0x7f9cb41077e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:04.591 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.017+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 >> 172.21.15.103:0/2018515638 conn(0x7f9cb4101a60 msgr2=0x7f9cb4102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:04.591 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.017+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 shutdown_connections 2024-09-05T01:53:04.591 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.017+0000 7f9cb8bdc640 1 -- 172.21.15.103:0/2018515638 wait complete. 2024-09-05T01:53:04.591 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 5 is available 2024-09-05T01:53:04.592 INFO:teuthology.orchestra.run.smithi103.stdout:Setting orchestrator backend to cephadm... 2024-09-05T01:53:05.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:05 smithi103 ceph-mon[30902]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-05T01:53:05.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:05 smithi103 ceph-mon[30902]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-05T01:53:05.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:05 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:05.135 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:04.891+0000 7fe959e7b640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:01:53:04] ENGINE Error in HTTPServer.serve 2024-09-05T01:53:05.135 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: Traceback (most recent call last): 2024-09-05T01:53:05.135 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T01:53:05.135 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: self._connections.run(self.expiration_interval) 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: self._run(expiration_interval) 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: s = self.context.wrap_socket( 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: return self.sslsocket_class._create( 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T01:53:05.136 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: self.do_handshake() 2024-09-05T01:53:05.137 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T01:53:05.137 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: self._sslobj.do_handshake() 2024-09-05T01:53:05.137 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T01:53:05.137 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:04 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:05.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.942+0000 7ff2194eb640 1 Processor -- start 2024-09-05T01:53:05.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.942+0000 7ff2194eb640 1 -- start start 2024-09-05T01:53:05.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.943+0000 7ff2194eb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21410a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:05.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.943+0000 7ff2194eb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff214075720 con 0x7ff214107ef0 2024-09-05T01:53:05.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.943+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21410a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:05.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.943+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21410a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40740/0 (socket says 172.21.15.103:40740) 2024-09-05T01:53:05.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.943+0000 7ff212ffd640 1 -- 172.21.15.103:0/3087695668 learned_addr learned my addr 172.21.15.103:0/3087695668 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:05.707 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.943+0000 7ff212ffd640 1 -- 172.21.15.103:0/3087695668 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21410a820 con 0x7ff214107ef0 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.944+0000 7ff212ffd640 1 --2- 172.21.15.103:0/3087695668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21410a2e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff204009820 tx=0x7ff20402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dba29b79b29f9ce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.944+0000 7ff211ffb640 1 -- 172.21.15.103:0/3087695668 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff204037070 con 0x7ff214107ef0 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.944+0000 7ff211ffb640 1 -- 172.21.15.103:0/3087695668 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff2040328d0 con 0x7ff214107ef0 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.944+0000 7ff211ffb640 1 -- 172.21.15.103:0/3087695668 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2040307d0 con 0x7ff214107ef0 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.945+0000 7ff2194eb640 1 -- 172.21.15.103:0/3087695668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 msgr2=0x7ff21410a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.945+0000 7ff2194eb640 1 --2- 172.21.15.103:0/3087695668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21410a2e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff204009820 tx=0x7ff20402a270 comp rx=0 tx=0).stop 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.945+0000 7ff2194eb640 1 -- 172.21.15.103:0/3087695668 shutdown_connections 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.945+0000 7ff2194eb640 1 --2- 172.21.15.103:0/3087695668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21410a2e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.945+0000 7ff2194eb640 1 -- 172.21.15.103:0/3087695668 >> 172.21.15.103:0/3087695668 conn(0x7ff214101a50 msgr2=0x7ff214103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.945+0000 7ff2194eb640 1 -- 172.21.15.103:0/3087695668 shutdown_connections 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.945+0000 7ff2194eb640 1 -- 172.21.15.103:0/3087695668 wait complete. 2024-09-05T01:53:05.708 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.945+0000 7ff2194eb640 1 Processor -- start 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff2194eb640 1 -- start start 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff2194eb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff2194eb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff21410b7a0 con 0x7ff214107ef0 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff212ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40746/0 (socket says 172.21.15.103:40746) 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff212ffd640 1 -- 172.21.15.103:0/79300262 learned_addr learned my addr 172.21.15.103:0/79300262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff212ffd640 1 -- 172.21.15.103:0/79300262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21419b860 con 0x7ff214107ef0 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff212ffd640 1 --2- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21419b320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2040097f0 tx=0x7ff204030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff1fbfff640 1 -- 172.21.15.103:0/79300262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff204037040 con 0x7ff214107ef0 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21419baf0 con 0x7ff214107ef0 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff1fbfff640 1 -- 172.21.15.103:0/79300262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff20403b3f0 con 0x7ff214107ef0 2024-09-05T01:53:05.709 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.946+0000 7ff1fbfff640 1 -- 172.21.15.103:0/79300262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff20403a520 con 0x7ff214107ef0 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.947+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21419e7e0 con 0x7ff214107ef0 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.947+0000 7ff1fbfff640 1 -- 172.21.15.103:0/79300262 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7ff20403a710 con 0x7ff214107ef0 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.947+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff214108350 con 0x7ff214107ef0 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.948+0000 7ff1fbfff640 1 --2- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7ff1fc038f20 0x7ff1fc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.948+0000 7ff1fbfff640 1 -- 172.21.15.103:0/79300262 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff20406c000 con 0x7ff214107ef0 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.948+0000 7ff2127fc640 1 --2- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7ff1fc038f20 0x7ff1fc03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.948+0000 7ff2127fc640 1 --2- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7ff1fc038f20 0x7ff1fc03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2080098c0 tx=0x7ff208006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:04.952+0000 7ff1fbfff640 1 -- 172.21.15.103:0/79300262 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff204032820 con 0x7ff214107ef0 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.053+0000 7ff1fbfff640 1 -- 172.21.15.103:0/79300262 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff20402fd40 con 0x7ff214107ef0 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.133+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7ff214102d40 con 0x7ff1fc038f20 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.138+0000 7ff1fbfff640 1 -- 172.21.15.103:0/79300262 <== mgr.14118 v2:172.21.15.103:6800/2384335573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff214102d40 con 0x7ff1fc038f20 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7ff1fc038f20 msgr2=0x7ff1fc03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:05.710 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 --2- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7ff1fc038f20 0x7ff1fc03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2080098c0 tx=0x7ff208006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:05.711 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 msgr2=0x7ff21419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:05.711 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 --2- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21419b320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2040097f0 tx=0x7ff204030d00 comp rx=0 tx=0).stop 2024-09-05T01:53:05.711 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 shutdown_connections 2024-09-05T01:53:05.711 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 --2- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7ff1fc038f20 0x7ff1fc03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:05.711 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 --2- 172.21.15.103:0/79300262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214107ef0 0x7ff21419b320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:05.711 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 >> 172.21.15.103:0/79300262 conn(0x7ff214101a50 msgr2=0x7ff214102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:05.711 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 shutdown_connections 2024-09-05T01:53:05.711 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:05.140+0000 7ff2194eb640 1 -- 172.21.15.103:0/79300262 wait complete. 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:06 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:04] ENGINE Bus STARTING 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:06 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:04] ENGINE Serving on https://172.21.15.103:7150 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:06 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:04] ENGINE Error in HTTPServer.serve 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T01:53:06.251 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:06 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:04] ENGINE Serving on http://172.21.15.103:8765 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:06 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:04] ENGINE Bus STARTED 2024-09-05T01:53:06.252 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:06 smithi103 ceph-mon[30902]: mgrmap e8: x(active, since 2s) 2024-09-05T01:53:06.253 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:06 smithi103 ceph-mon[30902]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:06.253 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:06 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:06.253 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:06 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:06.798 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-05T01:53:06.798 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.071+0000 7eff634c3640 1 Processor -- start 2024-09-05T01:53:06.798 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.071+0000 7eff634c3640 1 -- start start 2024-09-05T01:53:06.798 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.072+0000 7eff634c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.072+0000 7eff634c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c106ce0 con 0x7eff5c106310 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.072+0000 7eff6125f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.072+0000 7eff6125f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40758/0 (socket says 172.21.15.103:40758) 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.072+0000 7eff6125f640 1 -- 172.21.15.103:0/3271199352 learned_addr learned my addr 172.21.15.103:0/3271199352 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.072+0000 7eff6125f640 1 -- 172.21.15.103:0/3271199352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c107510 con 0x7eff5c106310 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.073+0000 7eff6125f640 1 --2- 172.21.15.103:0/3271199352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c106710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7eff58009820 tx=0x7eff5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3b4fcfc2fd4335 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.073+0000 7eff53fff640 1 -- 172.21.15.103:0/3271199352 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff58037070 con 0x7eff5c106310 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.073+0000 7eff53fff640 1 -- 172.21.15.103:0/3271199352 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7eff580328d0 con 0x7eff5c106310 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.073+0000 7eff53fff640 1 -- 172.21.15.103:0/3271199352 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff580307d0 con 0x7eff5c106310 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.073+0000 7eff634c3640 1 -- 172.21.15.103:0/3271199352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 msgr2=0x7eff5c106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.073+0000 7eff634c3640 1 --2- 172.21.15.103:0/3271199352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c106710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7eff58009820 tx=0x7eff5802a270 comp rx=0 tx=0).stop 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.073+0000 7eff634c3640 1 -- 172.21.15.103:0/3271199352 shutdown_connections 2024-09-05T01:53:06.799 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.073+0000 7eff634c3640 1 --2- 172.21.15.103:0/3271199352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c106710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.073+0000 7eff634c3640 1 -- 172.21.15.103:0/3271199352 >> 172.21.15.103:0/3271199352 conn(0x7eff5c101a60 msgr2=0x7eff5c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.074+0000 7eff634c3640 1 -- 172.21.15.103:0/3271199352 shutdown_connections 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.074+0000 7eff634c3640 1 -- 172.21.15.103:0/3271199352 wait complete. 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.074+0000 7eff634c3640 1 Processor -- start 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.074+0000 7eff634c3640 1 -- start start 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.074+0000 7eff634c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.074+0000 7eff634c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c10a8c0 con 0x7eff5c106310 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.074+0000 7eff6125f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.074+0000 7eff6125f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40762/0 (socket says 172.21.15.103:40762) 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.074+0000 7eff6125f640 1 -- 172.21.15.103:0/2764458569 learned_addr learned my addr 172.21.15.103:0/2764458569 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.075+0000 7eff6125f640 1 -- 172.21.15.103:0/2764458569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c19b8f0 con 0x7eff5c106310 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.075+0000 7eff6125f640 1 --2- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c19b3b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7eff58030520 tx=0x7eff58032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:06.800 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.075+0000 7eff527fc640 1 -- 172.21.15.103:0/2764458569 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff58042070 con 0x7eff5c106310 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.075+0000 7eff527fc640 1 -- 172.21.15.103:0/2764458569 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7eff58030cc0 con 0x7eff5c106310 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.075+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c19bb80 con 0x7eff5c106310 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.075+0000 7eff527fc640 1 -- 172.21.15.103:0/2764458569 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff58037040 con 0x7eff5c106310 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.075+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c19e870 con 0x7eff5c106310 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.076+0000 7eff527fc640 1 -- 172.21.15.103:0/2764458569 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7eff58045070 con 0x7eff5c106310 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.076+0000 7eff527fc640 1 --2- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7eff48039080 0x7eff4803b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.076+0000 7eff527fc640 1 -- 172.21.15.103:0/2764458569 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7eff5806c040 con 0x7eff5c106310 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.076+0000 7eff60a5e640 1 --2- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7eff48039080 0x7eff4803b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.076+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff5c10aa40 con 0x7eff5c106310 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.076+0000 7eff60a5e640 1 --2- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7eff48039080 0x7eff4803b540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0098c0 tx=0x7eff4c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.081+0000 7eff527fc640 1 -- 172.21.15.103:0/2764458569 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff5803caa0 con 0x7eff5c106310 2024-09-05T01:53:06.801 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.249+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7eff5c102cf0 con 0x7eff48039080 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.250+0000 7eff527fc640 1 -- 172.21.15.103:0/2764458569 <== mgr.14118 v2:172.21.15.103:6800/2384335573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7eff5c102cf0 con 0x7eff48039080 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.251+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7eff48039080 msgr2=0x7eff4803b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.251+0000 7eff634c3640 1 --2- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7eff48039080 0x7eff4803b540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0098c0 tx=0x7eff4c006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.251+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 msgr2=0x7eff5c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.251+0000 7eff634c3640 1 --2- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c19b3b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7eff58030520 tx=0x7eff58032dd0 comp rx=0 tx=0).stop 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.252+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 shutdown_connections 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.252+0000 7eff634c3640 1 --2- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7eff48039080 0x7eff4803b540 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.252+0000 7eff634c3640 1 --2- 172.21.15.103:0/2764458569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff5c106310 0x7eff5c19b3b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.252+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 >> 172.21.15.103:0/2764458569 conn(0x7eff5c101a60 msgr2=0x7eff5c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.252+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 shutdown_connections 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:06.252+0000 7eff634c3640 1 -- 172.21.15.103:0/2764458569 wait complete. 2024-09-05T01:53:06.802 INFO:teuthology.orchestra.run.smithi103.stdout:Generating ssh key... 2024-09-05T01:53:07.383 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-mon[30902]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:07.649 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: Generating public/private rsa key pair. 2024-09-05T01:53:07.649 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: Your identification has been saved in /tmp/tmpgx5llnpn/key 2024-09-05T01:53:07.649 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: Your public key has been saved in /tmp/tmpgx5llnpn/key.pub 2024-09-05T01:53:07.649 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: The key fingerprint is: 2024-09-05T01:53:07.649 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: SHA256:btT5BmSWzKuAjyAPSgUvwHM8qdTbT3gwqzXVt7Pg034 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:53:07.649 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: The key's randomart image is: 2024-09-05T01:53:07.649 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: +---[RSA 3072]----+ 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: |o.o . . | 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: |.+o* o . + o | 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: |..+o+ * O . | 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: | .o. B o * = | 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: |o.. + * S B o | 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: |o+ o o = + = | 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: |. . . . + o o | 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: | . o E | 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: | . | 2024-09-05T01:53:07.650 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:07 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: +----[SHA256]-----+ 2024-09-05T01:53:08.062 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.204+0000 7f458e8f0640 1 Processor -- start 2024-09-05T01:53:08.063 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.204+0000 7f458e8f0640 1 -- start start 2024-09-05T01:53:08.063 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.204+0000 7f458e8f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4588107440 0x7f4588107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:08.063 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.204+0000 7f458e8f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4588107e10 con 0x7f4588107440 2024-09-05T01:53:08.063 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.204+0000 7f4587fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4588107440 0x7f4588107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:08.063 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.204+0000 7f4587fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4588107440 0x7f4588107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40774/0 (socket says 172.21.15.103:40774) 2024-09-05T01:53:08.063 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.204+0000 7f4587fff640 1 -- 172.21.15.103:0/669149586 learned_addr learned my addr 172.21.15.103:0/669149586 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:08.063 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.204+0000 7f4587fff640 1 -- 172.21.15.103:0/669149586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4588108640 con 0x7f4588107440 2024-09-05T01:53:08.064 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.205+0000 7f4587fff640 1 --2- 172.21.15.103:0/669149586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4588107440 0x7f4588107840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4578009820 tx=0x7f457802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499d3138b7a4482 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:08.064 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.205+0000 7f4586ffd640 1 -- 172.21.15.103:0/669149586 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4578037070 con 0x7f4588107440 2024-09-05T01:53:08.064 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.205+0000 7f4586ffd640 1 -- 172.21.15.103:0/669149586 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f45780328d0 con 0x7f4588107440 2024-09-05T01:53:08.064 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.205+0000 7f4586ffd640 1 -- 172.21.15.103:0/669149586 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45780307d0 con 0x7f4588107440 2024-09-05T01:53:08.064 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.206+0000 7f458e8f0640 1 -- 172.21.15.103:0/669149586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4588107440 msgr2=0x7f4588107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:08.064 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.206+0000 7f458e8f0640 1 --2- 172.21.15.103:0/669149586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4588107440 0x7f4588107840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4578009820 tx=0x7f457802a270 comp rx=0 tx=0).stop 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.206+0000 7f458e8f0640 1 -- 172.21.15.103:0/669149586 shutdown_connections 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.206+0000 7f458e8f0640 1 --2- 172.21.15.103:0/669149586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4588107440 0x7f4588107840 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.206+0000 7f458e8f0640 1 -- 172.21.15.103:0/669149586 >> 172.21.15.103:0/669149586 conn(0x7f4588102bb0 msgr2=0x7f4588105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.206+0000 7f458e8f0640 1 -- 172.21.15.103:0/669149586 shutdown_connections 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.206+0000 7f458e8f0640 1 -- 172.21.15.103:0/669149586 wait complete. 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.207+0000 7f458e8f0640 1 Processor -- start 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.207+0000 7f458e8f0640 1 -- start start 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.207+0000 7f458e8f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f458807dad0 0x7f458807c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.207+0000 7f458e8f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f458810b9f0 con 0x7f458807dad0 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.207+0000 7f4587fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f458807dad0 0x7f458807c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:08.065 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.207+0000 7f4587fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f458807dad0 0x7f458807c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40786/0 (socket says 172.21.15.103:40786) 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.207+0000 7f4587fff640 1 -- 172.21.15.103:0/32196052 learned_addr learned my addr 172.21.15.103:0/32196052 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.207+0000 7f4587fff640 1 -- 172.21.15.103:0/32196052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458807def0 con 0x7f458807dad0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.208+0000 7f4587fff640 1 --2- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f458807dad0 0x7f458807c240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45780097f0 tx=0x7f4578033cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.208+0000 7f45857fa640 1 -- 172.21.15.103:0/32196052 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4578037040 con 0x7f458807dad0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.208+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458807c780 con 0x7f458807dad0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.208+0000 7f45857fa640 1 -- 172.21.15.103:0/32196052 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4578033db0 con 0x7f458807dad0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.208+0000 7f45857fa640 1 -- 172.21.15.103:0/32196052 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f457803b580 con 0x7f458807dad0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.208+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458807cc60 con 0x7f458807dad0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.208+0000 7f45857fa640 1 -- 172.21.15.103:0/32196052 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4578044030 con 0x7f458807dad0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.209+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458810bb70 con 0x7f458807dad0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.209+0000 7f45857fa640 1 --2- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f4570039030 0x7f457003b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.209+0000 7f45857fa640 1 -- 172.21.15.103:0/32196052 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4578042070 con 0x7f458807dad0 2024-09-05T01:53:08.066 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.209+0000 7f45877fe640 1 --2- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f4570039030 0x7f457003b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:08.067 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.209+0000 7f45877fe640 1 --2- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f4570039030 0x7f457003b4f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f457c0098c0 tx=0x7f457c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:08.067 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.213+0000 7f45857fa640 1 -- 172.21.15.103:0/32196052 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f457804b060 con 0x7f458807dad0 2024-09-05T01:53:08.067 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.381+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f4588103cd0 con 0x7f4570039030 2024-09-05T01:53:08.067 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.570+0000 7f45857fa640 1 -- 172.21.15.103:0/32196052 <== mgr.14118 v2:172.21.15.103:6800/2384335573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4588103cd0 con 0x7f4570039030 2024-09-05T01:53:08.067 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f4570039030 msgr2=0x7f457003b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:08.067 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 --2- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f4570039030 0x7f457003b4f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f457c0098c0 tx=0x7f457c006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:08.067 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f458807dad0 msgr2=0x7f458807c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:08.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 --2- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f458807dad0 0x7f458807c240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45780097f0 tx=0x7f4578033cc0 comp rx=0 tx=0).stop 2024-09-05T01:53:08.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 shutdown_connections 2024-09-05T01:53:08.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 --2- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f4570039030 0x7f457003b4f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:08.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 --2- 172.21.15.103:0/32196052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f458807dad0 0x7f458807c240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:08.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 >> 172.21.15.103:0/32196052 conn(0x7f4588102bb0 msgr2=0x7f45881035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:08.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 shutdown_connections 2024-09-05T01:53:08.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:07.572+0000 7f458e8f0640 1 -- 172.21.15.103:0/32196052 wait complete. 2024-09-05T01:53:08.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:08 smithi103 ceph-mon[30902]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:08.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:08 smithi103 ceph-mon[30902]: Generating ssh key... 2024-09-05T01:53:08.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:08 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:08.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:08 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:09.210 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEdzahn1JOPLH1n3QJZdpCt/MyHfXbgCZcNTi4kRsIRlxv67dvKSJE7Y4quAESW2hBsLi5eRKHVGQ1nVSyDJj7AOBtObngHdRnGwblW5jaFdO4cbOcT8Zm3mfZAZHJlab0xQcBO0bzHCJasMNW5VO/2o5xHciRTbOwu71khkogI978EHIX2Ne2UFfPhS8PK+Vpn8gA5XiY7zlxKz/LmFDrDDun3GKC7wq6njDJ7xLnBS2ayZm81QiSmp7yYPx5IOsBGzrLTadJJBkCxqcTlNR+ug9BZIfCQODYLARwdlISUk3d6yrtJ53ziTLH3Mc4x0uGa54I4CMiIV6tAchvDrKNK8eoJWoiPP0ZW5xMdc+0HWU6yt2P04CvyFZLQx/HPmjmBLgJ5bEgopW4Nhvl0DOPuTtKBg/tK3ol2N/uIDvdPWjLWMCvfQpXizbwWM010/PX8RuoEcr9v+1KMk3M+e53aKjyx6qBqcpRaIzdJ7cNjLe6YF3gKomP3xW4Izmnwd8= ceph-695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.442+0000 7f85472a2640 1 Processor -- start 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.442+0000 7f85472a2640 1 -- start start 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.442+0000 7f85472a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854007c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.442+0000 7f85472a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f854007c850 con 0x7f854007de20 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.442+0000 7f854503e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854007c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.442+0000 7f854503e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854007c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40792/0 (socket says 172.21.15.103:40792) 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.442+0000 7f854503e640 1 -- 172.21.15.103:0/2891074241 learned_addr learned my addr 172.21.15.103:0/2891074241 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.443+0000 7f854503e640 1 -- 172.21.15.103:0/2891074241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854007c9d0 con 0x7f854007de20 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.443+0000 7f854503e640 1 --2- 172.21.15.103:0/2891074241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854007c280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f853c009820 tx=0x7f853c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9535865b88b1cc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f8537fff640 1 -- 172.21.15.103:0/2891074241 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f853c037070 con 0x7f854007de20 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f8537fff640 1 -- 172.21.15.103:0/2891074241 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f853c0328d0 con 0x7f854007de20 2024-09-05T01:53:09.211 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f8537fff640 1 -- 172.21.15.103:0/2891074241 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f853c0307d0 con 0x7f854007de20 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f85472a2640 1 -- 172.21.15.103:0/2891074241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 msgr2=0x7f854007c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f85472a2640 1 --2- 172.21.15.103:0/2891074241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854007c280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f853c009820 tx=0x7f853c02a270 comp rx=0 tx=0).stop 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f85472a2640 1 -- 172.21.15.103:0/2891074241 shutdown_connections 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f85472a2640 1 --2- 172.21.15.103:0/2891074241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854007c280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f85472a2640 1 -- 172.21.15.103:0/2891074241 >> 172.21.15.103:0/2891074241 conn(0x7f8540102bc0 msgr2=0x7f8540105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f85472a2640 1 -- 172.21.15.103:0/2891074241 shutdown_connections 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.444+0000 7f85472a2640 1 -- 172.21.15.103:0/2891074241 wait complete. 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.445+0000 7f85472a2640 1 Processor -- start 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.445+0000 7f85472a2640 1 -- start start 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.445+0000 7f85472a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.445+0000 7f85472a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f854010da20 con 0x7f854007de20 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.445+0000 7f854503e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:09.212 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.445+0000 7f854503e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854019d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40798/0 (socket says 172.21.15.103:40798) 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.445+0000 7f854503e640 1 -- 172.21.15.103:0/2873644204 learned_addr learned my addr 172.21.15.103:0/2873644204 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.445+0000 7f854503e640 1 -- 172.21.15.103:0/2873644204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854019da00 con 0x7f854007de20 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.446+0000 7f854503e640 1 --2- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854019d4c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f853c032e10 tx=0x7f853c032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.446+0000 7f85367fc640 1 -- 172.21.15.103:0/2873644204 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f853c042070 con 0x7f854007de20 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.446+0000 7f85367fc640 1 -- 172.21.15.103:0/2873644204 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f853c030c80 con 0x7f854007de20 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.446+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854019dc90 con 0x7f854007de20 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.446+0000 7f85367fc640 1 -- 172.21.15.103:0/2873644204 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f853c037040 con 0x7f854007de20 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.446+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401a0980 con 0x7f854007de20 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.447+0000 7f85367fc640 1 -- 172.21.15.103:0/2873644204 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f853c045070 con 0x7f854007de20 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.447+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854007c300 con 0x7f854007de20 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.447+0000 7f85367fc640 1 --2- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f852c039080 0x7f852c03b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.447+0000 7f85367fc640 1 -- 172.21.15.103:0/2873644204 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f853c06bf00 con 0x7f854007de20 2024-09-05T01:53:09.213 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.447+0000 7f854483d640 1 --2- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f852c039080 0x7f852c03b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.447+0000 7f854483d640 1 --2- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f852c039080 0x7f852c03b540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f85300098c0 tx=0x7f8530006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.451+0000 7f85367fc640 1 -- 172.21.15.103:0/2873644204 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f853c045360 con 0x7f854007de20 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.618+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8540103d90 con 0x7f852c039080 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.619+0000 7f85367fc640 1 -- 172.21.15.103:0/2873644204 <== mgr.14118 v2:172.21.15.103:6800/2384335573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8540103d90 con 0x7f852c039080 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f852c039080 msgr2=0x7f852c03b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 --2- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f852c039080 0x7f852c03b540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f85300098c0 tx=0x7f8530006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 msgr2=0x7f854019d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 --2- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854019d4c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f853c032e10 tx=0x7f853c032e40 comp rx=0 tx=0).stop 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 shutdown_connections 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 --2- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f852c039080 0x7f852c03b540 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 --2- 172.21.15.103:0/2873644204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854007de20 0x7f854019d4c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 >> 172.21.15.103:0/2873644204 conn(0x7f8540102bc0 msgr2=0x7f8540103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 shutdown_connections 2024-09-05T01:53:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:08.621+0000 7f85472a2640 1 -- 172.21.15.103:0/2873644204 wait complete. 2024-09-05T01:53:09.215 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-05T01:53:09.215 INFO:teuthology.orchestra.run.smithi103.stdout:Adding key to root@localhost authorized_keys... 2024-09-05T01:53:09.215 INFO:teuthology.orchestra.run.smithi103.stdout:Adding host smithi103... 2024-09-05T01:53:09.739 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:09 smithi103 ceph-mon[30902]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:10.770 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:10 smithi103 ceph-mon[30902]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:11.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:11 smithi103 ceph-mon[30902]: Deploying cephadm binary to smithi103 2024-09-05T01:53:13.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Added host 'smithi103' with addr '172.21.15.103' 2024-09-05T01:53:13.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.559+0000 7f389c879640 1 Processor -- start 2024-09-05T01:53:13.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.559+0000 7f389c879640 1 -- start start 2024-09-05T01:53:13.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.559+0000 7f389c879640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:13.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.559+0000 7f389c879640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f389807c870 con 0x7f389807de40 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.559+0000 7f389659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.559+0000 7f389659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45024/0 (socket says 172.21.15.103:45024) 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.559+0000 7f389659c640 1 -- 172.21.15.103:0/1363379870 learned_addr learned my addr 172.21.15.103:0/1363379870 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.559+0000 7f389659c640 1 -- 172.21.15.103:0/1363379870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389807c9f0 con 0x7f389807de40 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.560+0000 7f389659c640 1 --2- 172.21.15.103:0/1363379870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389807c2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3880009820 tx=0x7f388002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc8035d2b8a30b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.560+0000 7f389559a640 1 -- 172.21.15.103:0/1363379870 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3880037070 con 0x7f389807de40 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.560+0000 7f389559a640 1 -- 172.21.15.103:0/1363379870 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f38800328d0 con 0x7f389807de40 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.560+0000 7f389559a640 1 -- 172.21.15.103:0/1363379870 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38800307d0 con 0x7f389807de40 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.561+0000 7f389c879640 1 -- 172.21.15.103:0/1363379870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 msgr2=0x7f389807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.561+0000 7f389c879640 1 --2- 172.21.15.103:0/1363379870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389807c2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3880009820 tx=0x7f388002a270 comp rx=0 tx=0).stop 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.561+0000 7f389c879640 1 -- 172.21.15.103:0/1363379870 shutdown_connections 2024-09-05T01:53:13.462 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.561+0000 7f389c879640 1 --2- 172.21.15.103:0/1363379870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389807c2a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.561+0000 7f389c879640 1 -- 172.21.15.103:0/1363379870 >> 172.21.15.103:0/1363379870 conn(0x7f3898102bc0 msgr2=0x7f3898105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.561+0000 7f389c879640 1 -- 172.21.15.103:0/1363379870 shutdown_connections 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.561+0000 7f389c879640 1 -- 172.21.15.103:0/1363379870 wait complete. 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.562+0000 7f389c879640 1 Processor -- start 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.562+0000 7f389c879640 1 -- start start 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.562+0000 7f389c879640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.562+0000 7f389c879640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f389810da60 con 0x7f389807de40 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.562+0000 7f389659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.562+0000 7f389659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45032/0 (socket says 172.21.15.103:45032) 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.562+0000 7f389659c640 1 -- 172.21.15.103:0/330480022 learned_addr learned my addr 172.21.15.103:0/330480022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.562+0000 7f389659c640 1 -- 172.21.15.103:0/330480022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389819da10 con 0x7f389807de40 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.562+0000 7f389659c640 1 --2- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389819d4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3880032e10 tx=0x7f3880032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:13.463 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.563+0000 7f388f7fe640 1 -- 172.21.15.103:0/330480022 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3880042070 con 0x7f389807de40 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.563+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389819dca0 con 0x7f389807de40 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.563+0000 7f388f7fe640 1 -- 172.21.15.103:0/330480022 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3880030c80 con 0x7f389807de40 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.563+0000 7f388f7fe640 1 -- 172.21.15.103:0/330480022 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3880037040 con 0x7f389807de40 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.563+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38981a0990 con 0x7f389807de40 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.563+0000 7f388f7fe640 1 -- 172.21.15.103:0/330480022 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3880045070 con 0x7f389807de40 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.564+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389807c320 con 0x7f389807de40 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.564+0000 7f388f7fe640 1 --2- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f3884039030 0x7f388403b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.564+0000 7f388f7fe640 1 -- 172.21.15.103:0/330480022 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f388006c170 con 0x7f389807de40 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.564+0000 7f3895d9b640 1 --2- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f3884039030 0x7f388403b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.564+0000 7f3895d9b640 1 --2- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f3884039030 0x7f388403b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38880098c0 tx=0x7f3888006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.568+0000 7f388f7fe640 1 -- 172.21.15.103:0/330480022 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3880045360 con 0x7f389807de40 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:09.738+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}) -- 0x7f3898103d40 con 0x7f3884039030 2024-09-05T01:53:13.464 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.864+0000 7f388f7fe640 1 -- 172.21.15.103:0/330480022 <== mgr.14118 v2:172.21.15.103:6800/2384335573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3898103d40 con 0x7f3884039030 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f3884039030 msgr2=0x7f388403b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 --2- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f3884039030 0x7f388403b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38880098c0 tx=0x7f3888006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 msgr2=0x7f389819d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 --2- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389819d4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3880032e10 tx=0x7f3880032e40 comp rx=0 tx=0).stop 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 shutdown_connections 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 --2- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f3884039030 0x7f388403b4f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 --2- 172.21.15.103:0/330480022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f389807de40 0x7f389819d4d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 >> 172.21.15.103:0/330480022 conn(0x7f3898102bc0 msgr2=0x7f3898103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 shutdown_connections 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:12.867+0000 7f389c879640 1 -- 172.21.15.103:0/330480022 wait complete. 2024-09-05T01:53:13.465 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mon service... 2024-09-05T01:53:14.059 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:13 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:14.059 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:13 smithi103 ceph-mon[30902]: Added host smithi103 2024-09-05T01:53:14.059 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:13 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-05T01:53:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.868+0000 7f1084146640 1 Processor -- start 2024-09-05T01:53:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.869+0000 7f1084146640 1 -- start start 2024-09-05T01:53:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.869+0000 7f1084146640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.869+0000 7f1084146640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f107c106480 con 0x7f107c105ab0 2024-09-05T01:53:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.869+0000 7f1081ee2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.869+0000 7f1081ee2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45044/0 (socket says 172.21.15.103:45044) 2024-09-05T01:53:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.869+0000 7f1081ee2640 1 -- 172.21.15.103:0/1591191203 learned_addr learned my addr 172.21.15.103:0/1591191203 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.869+0000 7f1081ee2640 1 -- 172.21.15.103:0/1591191203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c106cb0 con 0x7f107c105ab0 2024-09-05T01:53:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.870+0000 7f1081ee2640 1 --2- 172.21.15.103:0/1591191203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c105eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f107400a9c0 tx=0x7f107402b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f670e215007a93c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.870+0000 7f1080ee0640 1 -- 172.21.15.103:0/1591191203 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f107400b040 con 0x7f107c105ab0 2024-09-05T01:53:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.870+0000 7f1080ee0640 1 -- 172.21.15.103:0/1591191203 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1074030ad0 con 0x7f107c105ab0 2024-09-05T01:53:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.870+0000 7f1080ee0640 1 -- 172.21.15.103:0/1591191203 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1074032be0 con 0x7f107c105ab0 2024-09-05T01:53:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.871+0000 7f1084146640 1 -- 172.21.15.103:0/1591191203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 msgr2=0x7f107c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.871+0000 7f1084146640 1 --2- 172.21.15.103:0/1591191203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c105eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f107400a9c0 tx=0x7f107402b610 comp rx=0 tx=0).stop 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.871+0000 7f1084146640 1 -- 172.21.15.103:0/1591191203 shutdown_connections 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.871+0000 7f1084146640 1 --2- 172.21.15.103:0/1591191203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c105eb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.871+0000 7f1084146640 1 -- 172.21.15.103:0/1591191203 >> 172.21.15.103:0/1591191203 conn(0x7f107c101200 msgr2=0x7f107c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.871+0000 7f1084146640 1 -- 172.21.15.103:0/1591191203 shutdown_connections 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.871+0000 7f1084146640 1 -- 172.21.15.103:0/1591191203 wait complete. 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.871+0000 7f1084146640 1 Processor -- start 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.871+0000 7f1084146640 1 -- start start 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f1084146640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f1084146640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f107c107430 con 0x7f107c105ab0 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f1081ee2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f1081ee2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45058/0 (socket says 172.21.15.103:45058) 2024-09-05T01:53:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f1081ee2640 1 -- 172.21.15.103:0/2802592824 learned_addr learned my addr 172.21.15.103:0/2802592824 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f1081ee2640 1 -- 172.21.15.103:0/2802592824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c19b920 con 0x7f107c105ab0 2024-09-05T01:53:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f1081ee2640 1 --2- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c19b3e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f107402bbd0 tx=0x7f1074030740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f107affd640 1 -- 172.21.15.103:0/2802592824 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1074041070 con 0x7f107c105ab0 2024-09-05T01:53:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f107affd640 1 -- 172.21.15.103:0/2802592824 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f107403bad0 con 0x7f107c105ab0 2024-09-05T01:53:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107c19bbb0 con 0x7f107c105ab0 2024-09-05T01:53:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f107affd640 1 -- 172.21.15.103:0/2802592824 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f107403a820 con 0x7f107c105ab0 2024-09-05T01:53:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.872+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107c19e8a0 con 0x7f107c105ab0 2024-09-05T01:53:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.873+0000 7f107affd640 1 -- 172.21.15.103:0/2802592824 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1074044070 con 0x7f107c105ab0 2024-09-05T01:53:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.873+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107c107a90 con 0x7f107c105ab0 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.873+0000 7f107affd640 1 --2- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f1064039080 0x7f106403b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.873+0000 7f107affd640 1 -- 172.21.15.103:0/2802592824 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f107406cc20 con 0x7f107c105ab0 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.874+0000 7f10816e1640 1 --2- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f1064039080 0x7f106403b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.874+0000 7f10816e1640 1 --2- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f1064039080 0x7f106403b540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f10700098c0 tx=0x7f1070006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:13.878+0000 7f107affd640 1 -- 172.21.15.103:0/2802592824 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1074044760 con 0x7f107c105ab0 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.057+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f107c02dab0 con 0x7f1064039080 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.062+0000 7f107affd640 1 -- 172.21.15.103:0/2802592824 <== mgr.14118 v2:172.21.15.103:6800/2384335573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f107c02dab0 con 0x7f1064039080 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.065+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f1064039080 msgr2=0x7f106403b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.066+0000 7f1084146640 1 --2- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f1064039080 0x7f106403b540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f10700098c0 tx=0x7f1070006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.066+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 msgr2=0x7f107c19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.066+0000 7f1084146640 1 --2- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c19b3e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f107402bbd0 tx=0x7f1074030740 comp rx=0 tx=0).stop 2024-09-05T01:53:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.066+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 shutdown_connections 2024-09-05T01:53:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.066+0000 7f1084146640 1 --2- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f1064039080 0x7f106403b540 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.066+0000 7f1084146640 1 --2- 172.21.15.103:0/2802592824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f107c105ab0 0x7f107c19b3e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.066+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 >> 172.21.15.103:0/2802592824 conn(0x7f107c101200 msgr2=0x7f107c192040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.066+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 shutdown_connections 2024-09-05T01:53:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:14.066+0000 7f1084146640 1 -- 172.21.15.103:0/2802592824 wait complete. 2024-09-05T01:53:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mgr service... 2024-09-05T01:53:15.188 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:15 smithi103 ceph-mon[30902]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:15.189 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:15 smithi103 ceph-mon[30902]: Saving service mon spec with placement count:5 2024-09-05T01:53:15.189 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:15 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:16.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-05T01:53:16.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.297+0000 7fbd46d44640 1 Processor -- start 2024-09-05T01:53:16.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.297+0000 7fbd46d44640 1 -- start start 2024-09-05T01:53:16.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.297+0000 7fbd46d44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd40107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:16.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.297+0000 7fbd46d44640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd40107e20 con 0x7fbd40107450 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.298+0000 7fbd44ae0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd40107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.298+0000 7fbd44ae0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd40107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45074/0 (socket says 172.21.15.103:45074) 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.298+0000 7fbd44ae0640 1 -- 172.21.15.103:0/2499702450 learned_addr learned my addr 172.21.15.103:0/2499702450 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.298+0000 7fbd44ae0640 1 -- 172.21.15.103:0/2499702450 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd40108650 con 0x7fbd40107450 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd44ae0640 1 --2- 172.21.15.103:0/2499702450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd40107850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009820 tx=0x7fbd3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88583bee102e7432 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd3f7fe640 1 -- 172.21.15.103:0/2499702450 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd34037070 con 0x7fbd40107450 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd3f7fe640 1 -- 172.21.15.103:0/2499702450 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbd340328d0 con 0x7fbd40107450 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd3f7fe640 1 -- 172.21.15.103:0/2499702450 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd340307d0 con 0x7fbd40107450 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd46d44640 1 -- 172.21.15.103:0/2499702450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 msgr2=0x7fbd40107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd46d44640 1 --2- 172.21.15.103:0/2499702450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd40107850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009820 tx=0x7fbd3402a270 comp rx=0 tx=0).stop 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd46d44640 1 -- 172.21.15.103:0/2499702450 shutdown_connections 2024-09-05T01:53:16.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd46d44640 1 --2- 172.21.15.103:0/2499702450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd40107850 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd46d44640 1 -- 172.21.15.103:0/2499702450 >> 172.21.15.103:0/2499702450 conn(0x7fbd40102bc0 msgr2=0x7fbd40105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd46d44640 1 -- 172.21.15.103:0/2499702450 shutdown_connections 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.299+0000 7fbd46d44640 1 -- 172.21.15.103:0/2499702450 wait complete. 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.300+0000 7fbd46d44640 1 Processor -- start 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.300+0000 7fbd46d44640 1 -- start start 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.300+0000 7fbd46d44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd4019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.300+0000 7fbd46d44640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4010ba00 con 0x7fbd40107450 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.300+0000 7fbd44ae0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd4019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.300+0000 7fbd44ae0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd4019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45084/0 (socket says 172.21.15.103:45084) 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.300+0000 7fbd44ae0640 1 -- 172.21.15.103:0/1696263196 learned_addr learned my addr 172.21.15.103:0/1696263196 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.301+0000 7fbd44ae0640 1 -- 172.21.15.103:0/1696263196 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4019b810 con 0x7fbd40107450 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.301+0000 7fbd44ae0640 1 --2- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd4019b2d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009940 tx=0x7fbd34032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.301+0000 7fbd3dffb640 1 -- 172.21.15.103:0/1696263196 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd34042070 con 0x7fbd40107450 2024-09-05T01:53:16.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.301+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4019baa0 con 0x7fbd40107450 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.301+0000 7fbd3dffb640 1 -- 172.21.15.103:0/1696263196 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbd34030c80 con 0x7fbd40107450 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.301+0000 7fbd3dffb640 1 -- 172.21.15.103:0/1696263196 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd34037040 con 0x7fbd40107450 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.301+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4019e790 con 0x7fbd40107450 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.302+0000 7fbd3dffb640 1 -- 172.21.15.103:0/1696263196 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbd34045070 con 0x7fbd40107450 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.302+0000 7fbd3dffb640 1 --2- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fbd30039030 0x7fbd3003b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.302+0000 7fbd3dffb640 1 -- 172.21.15.103:0/1696263196 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd3406bf80 con 0x7fbd40107450 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.302+0000 7fbd3ffff640 1 --2- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fbd30039030 0x7fbd3003b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.302+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4010bb80 con 0x7fbd40107450 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.303+0000 7fbd3ffff640 1 --2- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fbd30039030 0x7fbd3003b4f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0098c0 tx=0x7fbd2c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.307+0000 7fbd3dffb640 1 -- 172.21.15.103:0/1696263196 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd34045360 con 0x7fbd40107450 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.486+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbd40103d00 con 0x7fbd30039030 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.491+0000 7fbd3dffb640 1 -- 172.21.15.103:0/1696263196 <== mgr.14118 v2:172.21.15.103:6800/2384335573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbd40103d00 con 0x7fbd30039030 2024-09-05T01:53:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fbd30039030 msgr2=0x7fbd3003b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:16.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 --2- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fbd30039030 0x7fbd3003b4f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0098c0 tx=0x7fbd2c006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:16.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 msgr2=0x7fbd4019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:16.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 --2- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd4019b2d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009940 tx=0x7fbd34032e10 comp rx=0 tx=0).stop 2024-09-05T01:53:16.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 shutdown_connections 2024-09-05T01:53:16.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 --2- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fbd30039030 0x7fbd3003b4f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:16.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 --2- 172.21.15.103:0/1696263196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd40107450 0x7fbd4019b2d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:16.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 >> 172.21.15.103:0/1696263196 conn(0x7fbd40102bc0 msgr2=0x7fbd401035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:16.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 shutdown_connections 2024-09-05T01:53:16.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:15.493+0000 7fbd46d44640 1 -- 172.21.15.103:0/1696263196 wait complete. 2024-09-05T01:53:16.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:16 smithi103 ceph-mon[30902]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:16.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:16 smithi103 ceph-mon[30902]: Saving service mgr spec with placement count:2 2024-09-05T01:53:16.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:16 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:17.592 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.568+0000 7fed8cd9c640 1 Processor -- start 2024-09-05T01:53:17.593 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.568+0000 7fed8cd9c640 1 -- start start 2024-09-05T01:53:17.593 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.568+0000 7fed8cd9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:17.593 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.568+0000 7fed8cd9c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed88106cd0 con 0x7fed88106300 2024-09-05T01:53:17.593 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.569+0000 7fed8659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:17.593 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.569+0000 7fed8659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45088/0 (socket says 172.21.15.103:45088) 2024-09-05T01:53:17.593 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.569+0000 7fed8659c640 1 -- 172.21.15.103:0/1959193631 learned_addr learned my addr 172.21.15.103:0/1959193631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:17.593 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.569+0000 7fed8659c640 1 -- 172.21.15.103:0/1959193631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed88107500 con 0x7fed88106300 2024-09-05T01:53:17.593 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8659c640 1 --2- 172.21.15.103:0/1959193631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88106700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fed78009e80 tx=0x7fed7802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff53995b95ac9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:17.593 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8559a640 1 -- 172.21.15.103:0/1959193631 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed78036070 con 0x7fed88106300 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8559a640 1 -- 172.21.15.103:0/1959193631 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fed78031890 con 0x7fed88106300 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8559a640 1 -- 172.21.15.103:0/1959193631 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed78007980 con 0x7fed88106300 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8cd9c640 1 -- 172.21.15.103:0/1959193631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 msgr2=0x7fed88106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8cd9c640 1 --2- 172.21.15.103:0/1959193631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88106700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fed78009e80 tx=0x7fed7802a7a0 comp rx=0 tx=0).stop 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8cd9c640 1 -- 172.21.15.103:0/1959193631 shutdown_connections 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8cd9c640 1 --2- 172.21.15.103:0/1959193631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88106700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8cd9c640 1 -- 172.21.15.103:0/1959193631 >> 172.21.15.103:0/1959193631 conn(0x7fed88101a50 msgr2=0x7fed88103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8cd9c640 1 -- 172.21.15.103:0/1959193631 shutdown_connections 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.570+0000 7fed8cd9c640 1 -- 172.21.15.103:0/1959193631 wait complete. 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.571+0000 7fed8cd9c640 1 Processor -- start 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.571+0000 7fed8cd9c640 1 -- start start 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.571+0000 7fed8cd9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:17.594 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.571+0000 7fed8cd9c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed8810a8b0 con 0x7fed88106300 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.571+0000 7fed8659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.571+0000 7fed8659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88075340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45096/0 (socket says 172.21.15.103:45096) 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.571+0000 7fed8659c640 1 -- 172.21.15.103:0/559591526 learned_addr learned my addr 172.21.15.103:0/559591526 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.572+0000 7fed8659c640 1 -- 172.21.15.103:0/559591526 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8819e090 con 0x7fed88106300 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.572+0000 7fed8659c640 1 --2- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88075340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed78007bb0 tx=0x7fed78031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.572+0000 7fed777fe640 1 -- 172.21.15.103:0/559591526 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed78041020 con 0x7fed88106300 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.572+0000 7fed777fe640 1 -- 172.21.15.103:0/559591526 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fed7803c900 con 0x7fed88106300 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.572+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8819e260 con 0x7fed88106300 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.572+0000 7fed777fe640 1 -- 172.21.15.103:0/559591526 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed78036040 con 0x7fed88106300 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.572+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8819e680 con 0x7fed88106300 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.573+0000 7fed777fe640 1 -- 172.21.15.103:0/559591526 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fed78044020 con 0x7fed88106300 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.573+0000 7fed777fe640 1 --2- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fed70039030 0x7fed7003b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:17.595 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.573+0000 7fed777fe640 1 -- 172.21.15.103:0/559591526 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fed7806d5d0 con 0x7fed88106300 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.573+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8810aa30 con 0x7fed88106300 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.573+0000 7fed85d9b640 1 --2- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fed70039030 0x7fed7003b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.573+0000 7fed85d9b640 1 --2- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fed70039030 0x7fed7003b4f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098c0 tx=0x7fed7c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.578+0000 7fed777fe640 1 -- 172.21.15.103:0/559591526 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed7804b020 con 0x7fed88106300 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.748+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fed88106780 con 0x7fed88106300 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.750+0000 7fed777fe640 1 -- 172.21.15.103:0/559591526 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7fed78045a00 con 0x7fed88106300 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.752+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fed70039030 msgr2=0x7fed7003b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.752+0000 7fed8cd9c640 1 --2- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fed70039030 0x7fed7003b4f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098c0 tx=0x7fed7c006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.752+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 msgr2=0x7fed88075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.752+0000 7fed8cd9c640 1 --2- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88075340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed78007bb0 tx=0x7fed78031c70 comp rx=0 tx=0).stop 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.752+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 shutdown_connections 2024-09-05T01:53:17.596 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.752+0000 7fed8cd9c640 1 --2- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fed70039030 0x7fed7003b4f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:17.597 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.752+0000 7fed8cd9c640 1 --2- 172.21.15.103:0/559591526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed88106300 0x7fed88075340 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:17.597 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.752+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 >> 172.21.15.103:0/559591526 conn(0x7fed88101a50 msgr2=0x7fed881025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:17.597 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.753+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 shutdown_connections 2024-09-05T01:53:17.597 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:16.753+0000 7fed8cd9c640 1 -- 172.21.15.103:0/559591526 wait complete. 2024-09-05T01:53:17.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:17 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/559591526' entity='client.admin' 2024-09-05T01:53:17.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:17 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:18.774 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.991+0000 7fe0cf180640 1 Processor -- start 2024-09-05T01:53:18.774 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.991+0000 7fe0cf180640 1 -- start start 2024-09-05T01:53:18.774 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.991+0000 7fe0cf180640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c8107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:18.774 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.991+0000 7fe0cf180640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8107e20 con 0x7fe0c8107450 2024-09-05T01:53:18.774 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.991+0000 7fe0ccf1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c8107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:18.774 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.991+0000 7fe0ccf1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c8107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45106/0 (socket says 172.21.15.103:45106) 2024-09-05T01:53:18.774 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.991+0000 7fe0ccf1c640 1 -- 172.21.15.103:0/793040246 learned_addr learned my addr 172.21.15.103:0/793040246 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:18.774 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.992+0000 7fe0ccf1c640 1 -- 172.21.15.103:0/793040246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c8108650 con 0x7fe0c8107450 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.992+0000 7fe0ccf1c640 1 --2- 172.21.15.103:0/793040246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c8107850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009820 tx=0x7fe0bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5008f12a35ff8cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.992+0000 7fe0c77fe640 1 -- 172.21.15.103:0/793040246 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0bc037070 con 0x7fe0c8107450 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.992+0000 7fe0c77fe640 1 -- 172.21.15.103:0/793040246 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0bc0328f0 con 0x7fe0c8107450 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.992+0000 7fe0c77fe640 1 -- 172.21.15.103:0/793040246 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0bc030740 con 0x7fe0c8107450 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.993+0000 7fe0cf180640 1 -- 172.21.15.103:0/793040246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 msgr2=0x7fe0c8107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.993+0000 7fe0cf180640 1 --2- 172.21.15.103:0/793040246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c8107850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009820 tx=0x7fe0bc02a270 comp rx=0 tx=0).stop 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.993+0000 7fe0cf180640 1 -- 172.21.15.103:0/793040246 shutdown_connections 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.993+0000 7fe0cf180640 1 --2- 172.21.15.103:0/793040246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c8107850 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.993+0000 7fe0cf180640 1 -- 172.21.15.103:0/793040246 >> 172.21.15.103:0/793040246 conn(0x7fe0c8102bc0 msgr2=0x7fe0c8105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.993+0000 7fe0cf180640 1 -- 172.21.15.103:0/793040246 shutdown_connections 2024-09-05T01:53:18.775 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.993+0000 7fe0cf180640 1 -- 172.21.15.103:0/793040246 wait complete. 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0cf180640 1 Processor -- start 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0cf180640 1 -- start start 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0cf180640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c807db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0cf180640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c810ba00 con 0x7fe0c8107450 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0ccf1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c807db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0ccf1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c807db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45112/0 (socket says 172.21.15.103:45112) 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0ccf1c640 1 -- 172.21.15.103:0/699311825 learned_addr learned my addr 172.21.15.103:0/699311825 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0ccf1c640 1 -- 172.21.15.103:0/699311825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c807e050 con 0x7fe0c8107450 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0ccf1c640 1 --2- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c807db10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009940 tx=0x7fe0bc030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0c5ffb640 1 -- 172.21.15.103:0/699311825 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0bc042070 con 0x7fe0c8107450 2024-09-05T01:53:18.776 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.994+0000 7fe0c5ffb640 1 -- 172.21.15.103:0/699311825 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0bc030e30 con 0x7fe0c8107450 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.995+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c807c240 con 0x7fe0c8107450 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.995+0000 7fe0c5ffb640 1 -- 172.21.15.103:0/699311825 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0bc037040 con 0x7fe0c8107450 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.995+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c807c780 con 0x7fe0c8107450 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.996+0000 7fe0c5ffb640 1 -- 172.21.15.103:0/699311825 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe0bc045070 con 0x7fe0c8107450 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.996+0000 7fe0c5ffb640 1 --2- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fe0b8039030 0x7fe0b803b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.996+0000 7fe0c5ffb640 1 -- 172.21.15.103:0/699311825 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe0bc06cf30 con 0x7fe0c8107450 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.996+0000 7fe0c7fff640 1 --2- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fe0b8039030 0x7fe0b803b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.996+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c810bb80 con 0x7fe0c8107450 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:17.996+0000 7fe0c7fff640 1 --2- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fe0b8039030 0x7fe0b803b4f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40098c0 tx=0x7fe0b4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.001+0000 7fe0c5ffb640 1 -- 172.21.15.103:0/699311825 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0bc046e00 con 0x7fe0c8107450 2024-09-05T01:53:18.777 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.194+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fe0c807cde0 con 0x7fe0c8107450 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.196+0000 7fe0c5ffb640 1 -- 172.21.15.103:0/699311825 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fe0bc045360 con 0x7fe0c8107450 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fe0b8039030 msgr2=0x7fe0b803b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 --2- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fe0b8039030 0x7fe0b803b4f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40098c0 tx=0x7fe0b4006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 msgr2=0x7fe0c807db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 --2- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c807db10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009940 tx=0x7fe0bc030cc0 comp rx=0 tx=0).stop 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 shutdown_connections 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 --2- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fe0b8039030 0x7fe0b803b4f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 --2- 172.21.15.103:0/699311825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c8107450 0x7fe0c807db10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 >> 172.21.15.103:0/699311825 conn(0x7fe0c8102bc0 msgr2=0x7fe0c8103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 shutdown_connections 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:18.199+0000 7fe0cf180640 1 -- 172.21.15.103:0/699311825 wait complete. 2024-09-05T01:53:18.778 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling the dashboard module... 2024-09-05T01:53:19.367 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:19 smithi103 ceph-mon[30902]: from='mgr.14118 172.21.15.103:0/2874628795' entity='mgr.x' 2024-09-05T01:53:19.367 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:19 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/699311825' entity='client.admin' 2024-09-05T01:53:20.503 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:20 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2633145483' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-05T01:53:20.503 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: ignoring --setuser ceph since I am not root 2024-09-05T01:53:20.503 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: ignoring --setgroup ceph since I am not root 2024-09-05T01:53:20.503 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:20.309+0000 7f825987e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:53:20.503 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:20.367+0000 7f825987e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:53:20.813 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:20.517+0000 7f825987e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:53:20.813 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:20.583+0000 7f825987e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:53:20.813 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:20.731+0000 7f825987e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:53:20.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.365+0000 7f9541693640 1 Processor -- start 2024-09-05T01:53:20.820 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.365+0000 7f9541693640 1 -- start start 2024-09-05T01:53:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.365+0000 7f9541693640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.365+0000 7f9541693640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f953c107e20 con 0x7f953c107450 2024-09-05T01:53:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.365+0000 7f953affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.365+0000 7f953affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34392/0 (socket says 172.21.15.103:34392) 2024-09-05T01:53:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.365+0000 7f953affd640 1 -- 172.21.15.103:0/691803543 learned_addr learned my addr 172.21.15.103:0/691803543 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.366+0000 7f953affd640 1 -- 172.21.15.103:0/691803543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c108650 con 0x7f953c107450 2024-09-05T01:53:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.366+0000 7f953affd640 1 --2- 172.21.15.103:0/691803543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c107850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f952c009820 tx=0x7f952c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f796ac9fabf00cc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.366+0000 7f9539ffb640 1 -- 172.21.15.103:0/691803543 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f952c037070 con 0x7f953c107450 2024-09-05T01:53:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.366+0000 7f9539ffb640 1 -- 172.21.15.103:0/691803543 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f952c0328f0 con 0x7f953c107450 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.366+0000 7f9539ffb640 1 -- 172.21.15.103:0/691803543 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f952c030740 con 0x7f953c107450 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.367+0000 7f9541693640 1 -- 172.21.15.103:0/691803543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 msgr2=0x7f953c107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.367+0000 7f9541693640 1 --2- 172.21.15.103:0/691803543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c107850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f952c009820 tx=0x7f952c02a270 comp rx=0 tx=0).stop 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.367+0000 7f9541693640 1 -- 172.21.15.103:0/691803543 shutdown_connections 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.367+0000 7f9541693640 1 --2- 172.21.15.103:0/691803543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c107850 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.367+0000 7f9541693640 1 -- 172.21.15.103:0/691803543 >> 172.21.15.103:0/691803543 conn(0x7f953c102bc0 msgr2=0x7f953c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.367+0000 7f9541693640 1 -- 172.21.15.103:0/691803543 shutdown_connections 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.367+0000 7f9541693640 1 -- 172.21.15.103:0/691803543 wait complete. 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.367+0000 7f9541693640 1 Processor -- start 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.368+0000 7f9541693640 1 -- start start 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.368+0000 7f9541693640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.368+0000 7f9541693640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f953c10ba00 con 0x7f953c107450 2024-09-05T01:53:20.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.368+0000 7f953affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.368+0000 7f953affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34398/0 (socket says 172.21.15.103:34398) 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.368+0000 7f953affd640 1 -- 172.21.15.103:0/2633145483 learned_addr learned my addr 172.21.15.103:0/2633145483 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.368+0000 7f953affd640 1 -- 172.21.15.103:0/2633145483 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c19b850 con 0x7f953c107450 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.368+0000 7f953affd640 1 --2- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c19b310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f952c009940 tx=0x7f952c030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.368+0000 7f9523fff640 1 -- 172.21.15.103:0/2633145483 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f952c042070 con 0x7f953c107450 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.369+0000 7f9523fff640 1 -- 172.21.15.103:0/2633145483 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f952c030e30 con 0x7f953c107450 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.369+0000 7f9523fff640 1 -- 172.21.15.103:0/2633145483 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f952c037040 con 0x7f953c107450 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.369+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953c19bae0 con 0x7f953c107450 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.369+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953c19e7d0 con 0x7f953c107450 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.369+0000 7f9523fff640 1 -- 172.21.15.103:0/2633145483 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f952c045070 con 0x7f953c107450 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.370+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953c10bb80 con 0x7f953c107450 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.370+0000 7f9523fff640 1 --2- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9524039080 0x7f952403b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:20.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.370+0000 7f9523fff640 1 -- 172.21.15.103:0/2633145483 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f952c06bfe0 con 0x7f953c107450 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.370+0000 7f953a7fc640 1 --2- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9524039080 0x7f952403b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.370+0000 7f953a7fc640 1 --2- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9524039080 0x7f952403b540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f95300098c0 tx=0x7f9530006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.374+0000 7f9523fff640 1 -- 172.21.15.103:0/2633145483 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f952c045360 con 0x7f953c107450 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:19.598+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f953c19e9a0 con 0x7f953c107450 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.153+0000 7f9523fff640 1 -- 172.21.15.103:0/2633145483 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f952c02ed90 con 0x7f953c107450 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.155+0000 7f9523fff640 1 -- 172.21.15.103:0/2633145483 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f952c03b330 con 0x7f953c107450 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.156+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9524039080 msgr2=0x7f952403b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.156+0000 7f9541693640 1 --2- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9524039080 0x7f952403b540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f95300098c0 tx=0x7f9530006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.156+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 msgr2=0x7f953c19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.156+0000 7f9541693640 1 --2- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c19b310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f952c009940 tx=0x7f952c030cc0 comp rx=0 tx=0).stop 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.156+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 shutdown_connections 2024-09-05T01:53:20.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.156+0000 7f9541693640 1 --2- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7f9524039080 0x7f952403b540 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:20.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.156+0000 7f9541693640 1 --2- 172.21.15.103:0/2633145483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107450 0x7f953c19b310 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:20.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.156+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 >> 172.21.15.103:0/2633145483 conn(0x7f953c102bc0 msgr2=0x7f953c103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:20.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.156+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 shutdown_connections 2024-09-05T01:53:20.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:20.157+0000 7f9541693640 1 -- 172.21.15.103:0/2633145483 wait complete. 2024-09-05T01:53:21.107 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:20.878+0000 7f825987e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:53:21.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:21 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2633145483' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-05T01:53:21.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:21 smithi103 ceph-mon[30902]: mgrmap e9: x(active, since 17s) 2024-09-05T01:53:21.392 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:21 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:21.204+0000 7f825987e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:53:21.392 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:21 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:21.266+0000 7f825987e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:53:21.648 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:21 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:21.471+0000 7f825987e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:53:22.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:53:22.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-05T01:53:22.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T01:53:22.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T01:53:22.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T01:53:22.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:53:22.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.183+0000 7fc3e7e51640 1 Processor -- start 2024-09-05T01:53:22.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.183+0000 7fc3e7e51640 1 -- start start 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.184+0000 7fc3e7e51640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e010a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.184+0000 7fc3e7e51640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e00757a0 con 0x7fc3e0107e70 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.184+0000 7fc3e5bed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e010a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.184+0000 7fc3e5bed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e010a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34434/0 (socket says 172.21.15.103:34434) 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.184+0000 7fc3e5bed640 1 -- 172.21.15.103:0/1012224975 learned_addr learned my addr 172.21.15.103:0/1012224975 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.184+0000 7fc3e5bed640 1 -- 172.21.15.103:0/1012224975 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e010a7a0 con 0x7fc3e0107e70 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.185+0000 7fc3e5bed640 1 --2- 172.21.15.103:0/1012224975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e010a260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009820 tx=0x7fc3d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87cb8e206acd7f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.185+0000 7fc3e4beb640 1 -- 172.21.15.103:0/1012224975 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d0037070 con 0x7fc3e0107e70 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.185+0000 7fc3e4beb640 1 -- 172.21.15.103:0/1012224975 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3d00328f0 con 0x7fc3e0107e70 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.185+0000 7fc3e4beb640 1 -- 172.21.15.103:0/1012224975 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d0030740 con 0x7fc3e0107e70 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.186+0000 7fc3e7e51640 1 -- 172.21.15.103:0/1012224975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 msgr2=0x7fc3e010a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:22.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.186+0000 7fc3e7e51640 1 --2- 172.21.15.103:0/1012224975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e010a260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009820 tx=0x7fc3d002a270 comp rx=0 tx=0).stop 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.186+0000 7fc3e7e51640 1 -- 172.21.15.103:0/1012224975 shutdown_connections 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.186+0000 7fc3e7e51640 1 --2- 172.21.15.103:0/1012224975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e010a260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.186+0000 7fc3e7e51640 1 -- 172.21.15.103:0/1012224975 >> 172.21.15.103:0/1012224975 conn(0x7fc3e0101a50 msgr2=0x7fc3e0103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.186+0000 7fc3e7e51640 1 -- 172.21.15.103:0/1012224975 shutdown_connections 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.186+0000 7fc3e7e51640 1 -- 172.21.15.103:0/1012224975 wait complete. 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.186+0000 7fc3e7e51640 1 Processor -- start 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.186+0000 7fc3e7e51640 1 -- start start 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3e7e51640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e01076f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3e7e51640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e010b820 con 0x7fc3e0107e70 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3e5bed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e01076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3e5bed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e01076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34444/0 (socket says 172.21.15.103:34444) 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3e5bed640 1 -- 172.21.15.103:0/3989002007 learned_addr learned my addr 172.21.15.103:0/3989002007 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:22.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3e5bed640 1 -- 172.21.15.103:0/3989002007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e0107c30 con 0x7fc3e0107e70 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3e5bed640 1 --2- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e01076f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00097f0 tx=0x7fc3d0030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3d6ffd640 1 -- 172.21.15.103:0/3989002007 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d0042070 con 0x7fc3e0107e70 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3d6ffd640 1 -- 172.21.15.103:0/3989002007 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3d0030e80 con 0x7fc3e0107e70 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e0105e20 con 0x7fc3e0107e70 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3d6ffd640 1 -- 172.21.15.103:0/3989002007 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d0037040 con 0x7fc3e0107e70 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.187+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e0106360 con 0x7fc3e0107e70 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.188+0000 7fc3d6ffd640 1 -- 172.21.15.103:0/3989002007 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc3d0045070 con 0x7fc3e0107e70 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.188+0000 7fc3d6ffd640 1 --2- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fc3cc0390e0 0x7fc3cc03b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.188+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e01082d0 con 0x7fc3e0107e70 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.189+0000 7fc3d6ffd640 1 -- 172.21.15.103:0/3989002007 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc3d006c0b0 con 0x7fc3e0107e70 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.189+0000 7fc3e53ec640 1 -- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fc3cc0390e0 msgr2=0x7fc3cc03b5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2384335573 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.189+0000 7fc3e53ec640 1 --2- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fc3cc0390e0 0x7fc3cc03b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:53:22.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.193+0000 7fc3d6ffd640 1 -- 172.21.15.103:0/3989002007 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3d0045520 con 0x7fc3e0107e70 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.390+0000 7fc3e53ec640 1 -- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fc3cc0390e0 msgr2=0x7fc3cc03b5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2384335573 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.390+0000 7fc3e53ec640 1 --2- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fc3cc0390e0 0x7fc3cc03b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.417+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc3e0106c20 con 0x7fc3e0107e70 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.417+0000 7fc3d6ffd640 1 -- 172.21.15.103:0/3989002007 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fc3d003add0 con 0x7fc3e0107e70 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.419+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fc3cc0390e0 msgr2=0x7fc3cc03b5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.419+0000 7fc3e7e51640 1 --2- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fc3cc0390e0 0x7fc3cc03b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.419+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 msgr2=0x7fc3e01076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.419+0000 7fc3e7e51640 1 --2- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e01076f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00097f0 tx=0x7fc3d0030c70 comp rx=0 tx=0).stop 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.419+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 shutdown_connections 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.419+0000 7fc3e7e51640 1 --2- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7fc3cc0390e0 0x7fc3cc03b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.419+0000 7fc3e7e51640 1 --2- 172.21.15.103:0/3989002007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e0107e70 0x7fc3e01076f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.419+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 >> 172.21.15.103:0/3989002007 conn(0x7fc3e0101a50 msgr2=0x7fc3e0102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:22.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.420+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 shutdown_connections 2024-09-05T01:53:22.058 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:21.420+0000 7fc3e7e51640 1 -- 172.21.15.103:0/3989002007 wait complete. 2024-09-05T01:53:22.058 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-05T01:53:22.058 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 9... 2024-09-05T01:53:22.347 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:22 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3989002007' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T01:53:22.633 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:22 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:22.375+0000 7f825987e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:53:22.633 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:22 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:22.446+0000 7f825987e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:53:23.398 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:23.150+0000 7f825987e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:53:23.398 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:23.264+0000 7f825987e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:53:23.679 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:23.404+0000 7f825987e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:53:23.679 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:23.464+0000 7f825987e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:53:23.679 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:23.610+0000 7f825987e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:53:23.679 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:23.677+0000 7f825987e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:53:23.959 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:23.737+0000 7f825987e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:53:23.959 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:23.894+0000 7f825987e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:53:23.959 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:23.957+0000 7f825987e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:53:24.223 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:24.024+0000 7f825987e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:53:24.223 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:24.105+0000 7f825987e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:53:24.223 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:24.163+0000 7f825987e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:53:24.223 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:24.221+0000 7f825987e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:53:25.398 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:24.999+0000 7f825987e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: Active manager daemon x restarted 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: Activating manager daemon x 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: osdmap e3: 0 total, 0 up, 0 in 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: mgrmap e10: x(active, starting, since 0.0033614s) 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: Manager daemon x is now available 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T01:53:25.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T01:53:26.745 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:26.483+0000 7f81bf348640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:01:53:26] ENGINE Error in HTTPServer.serve 2024-09-05T01:53:26.745 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: Traceback (most recent call last): 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: self._connections.run(self.expiration_interval) 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: self._run(expiration_interval) 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: s = self.context.wrap_socket( 2024-09-05T01:53:26.746 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T01:53:26.747 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: return self.sslsocket_class._create( 2024-09-05T01:53:26.747 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T01:53:26.747 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: self.do_handshake() 2024-09-05T01:53:26.747 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T01:53:26.747 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: self._sslobj.do_handshake() 2024-09-05T01:53:26.747 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T01:53:26.747 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 01:53:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T01:53:26.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:53:26.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-05T01:53:26.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T01:53:26.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:53:26.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.424+0000 7faf23f20640 1 Processor -- start 2024-09-05T01:53:26.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.424+0000 7faf23f20640 1 -- start start 2024-09-05T01:53:26.873 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.425+0000 7faf23f20640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c10a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.425+0000 7faf23f20640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c07bda0 con 0x7faf1c107f70 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.425+0000 7faf21cbc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c10a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.425+0000 7faf21cbc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c10a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34446/0 (socket says 172.21.15.103:34446) 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.425+0000 7faf21cbc640 1 -- 172.21.15.103:0/447801333 learned_addr learned my addr 172.21.15.103:0/447801333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.425+0000 7faf21cbc640 1 -- 172.21.15.103:0/447801333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c10a8a0 con 0x7faf1c107f70 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.426+0000 7faf21cbc640 1 --2- 172.21.15.103:0/447801333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c10a360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faf18009820 tx=0x7faf1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28be571d2d9ab92 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.426+0000 7faf20cba640 1 -- 172.21.15.103:0/447801333 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf18037070 con 0x7faf1c107f70 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.426+0000 7faf20cba640 1 -- 172.21.15.103:0/447801333 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf180328f0 con 0x7faf1c107f70 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.426+0000 7faf20cba640 1 -- 172.21.15.103:0/447801333 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf18030740 con 0x7faf1c107f70 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.427+0000 7faf23f20640 1 -- 172.21.15.103:0/447801333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 msgr2=0x7faf1c10a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.427+0000 7faf23f20640 1 --2- 172.21.15.103:0/447801333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c10a360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faf18009820 tx=0x7faf1802a270 comp rx=0 tx=0).stop 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.427+0000 7faf23f20640 1 -- 172.21.15.103:0/447801333 shutdown_connections 2024-09-05T01:53:26.874 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.427+0000 7faf23f20640 1 --2- 172.21.15.103:0/447801333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c10a360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.427+0000 7faf23f20640 1 -- 172.21.15.103:0/447801333 >> 172.21.15.103:0/447801333 conn(0x7faf1c101b10 msgr2=0x7faf1c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.427+0000 7faf23f20640 1 -- 172.21.15.103:0/447801333 shutdown_connections 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.427+0000 7faf23f20640 1 -- 172.21.15.103:0/447801333 wait complete. 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.428+0000 7faf23f20640 1 Processor -- start 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.428+0000 7faf23f20640 1 -- start start 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.428+0000 7faf23f20640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.428+0000 7faf23f20640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c10b7f0 con 0x7faf1c107f70 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.428+0000 7faf21cbc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.428+0000 7faf21cbc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c19d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34458/0 (socket says 172.21.15.103:34458) 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.428+0000 7faf21cbc640 1 -- 172.21.15.103:0/4176377119 learned_addr learned my addr 172.21.15.103:0/4176377119 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.429+0000 7faf21cbc640 1 -- 172.21.15.103:0/4176377119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c19da80 con 0x7faf1c107f70 2024-09-05T01:53:26.875 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.429+0000 7faf21cbc640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c19d540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf180327b0 tx=0x7faf180327e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.429+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf18042070 con 0x7faf1c107f70 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.429+0000 7faf23f20640 1 -- 172.21.15.103:0/4176377119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c19dd10 con 0x7faf1c107f70 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.429+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf18030d50 con 0x7faf1c107f70 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.429+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf18037040 con 0x7faf1c107f70 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.429+0000 7faf23f20640 1 -- 172.21.15.103:0/4176377119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c1a0a00 con 0x7faf1c107f70 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.430+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7faf18045070 con 0x7faf1c107f70 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.430+0000 7faf12ffd640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 0x7faf0803b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.430+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 --> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faf0803bc00 con 0x7faf08039030 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.430+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faf1806c1c0 con 0x7faf1c107f70 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.430+0000 7faf214bb640 1 -- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 msgr2=0x7faf0803b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2384335573 2024-09-05T01:53:26.876 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.430+0000 7faf214bb640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 0x7faf0803b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.631+0000 7faf214bb640 1 -- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 msgr2=0x7faf0803b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2384335573 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:22.631+0000 7faf214bb640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 0x7faf0803b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:23.032+0000 7faf214bb640 1 -- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 msgr2=0x7faf0803b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2384335573 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:23.032+0000 7faf214bb640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 0x7faf0803b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:23.833+0000 7faf214bb640 1 -- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 msgr2=0x7faf0803b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2384335573 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:23.833+0000 7faf214bb640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 0x7faf0803b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:25.002+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7faf1806ac70 con 0x7faf1c107f70 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:25.003+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 msgr2=0x7faf0803b4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:25.003+0000 7faf12ffd640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 0x7faf0803b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.004+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7faf1803b330 con 0x7faf1c107f70 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.004+0000 7faf12ffd640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faf0803caf0 0x7faf0803eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:26.877 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.004+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faf0803bc00 con 0x7faf0803caf0 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.004+0000 7faf214bb640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faf0803caf0 0x7faf0803eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.005+0000 7faf214bb640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faf0803caf0 0x7faf0803eee0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf0c003980 tx=0x7faf0c006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.005+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7faf0803bc00 con 0x7faf0803caf0 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.012+0000 7faf23f20640 1 -- 172.21.15.103:0/4176377119 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7faf1c1083d0 con 0x7faf0803caf0 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf12ffd640 1 -- 172.21.15.103:0/4176377119 <== mgr.14150 v2:172.21.15.103:6800/4267002703 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7faf1c1083d0 con 0x7faf0803caf0 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 -- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faf0803caf0 msgr2=0x7faf0803eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faf0803caf0 0x7faf0803eee0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf0c003980 tx=0x7faf0c006e10 comp rx=0 tx=0).stop 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 -- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 msgr2=0x7faf1c19d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c19d540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf180327b0 tx=0x7faf180327e0 comp rx=0 tx=0).stop 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 -- 172.21.15.103:0/4176377119 shutdown_connections 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faf0803caf0 0x7faf0803eee0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:6800/2384335573,v1:172.21.15.103:6801/2384335573] conn(0x7faf08039030 0x7faf0803b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:26.878 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 --2- 172.21.15.103:0/4176377119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf1c107f70 0x7faf1c19d540 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:26.879 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 -- 172.21.15.103:0/4176377119 >> 172.21.15.103:0/4176377119 conn(0x7faf1c101b10 msgr2=0x7faf1c1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:26.879 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 -- 172.21.15.103:0/4176377119 shutdown_connections 2024-09-05T01:53:26.879 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:26.013+0000 7faf23f20640 1 -- 172.21.15.103:0/4176377119 wait complete. 2024-09-05T01:53:26.879 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 9 is available 2024-09-05T01:53:26.879 INFO:teuthology.orchestra.run.smithi103.stdout:Generating a dashboard self-signed certificate... 2024-09-05T01:53:27.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:27 smithi103 ceph-mon[30902]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-05T01:53:27.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:27 smithi103 ceph-mon[30902]: mgrmap e11: x(active, since 1.00583s) 2024-09-05T01:53:27.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:27 smithi103 ceph-mon[30902]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-05T01:53:27.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:28.230 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-05T01:53:28.230 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.344+0000 7f6d0b691640 1 Processor -- start 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.345+0000 7f6d0b691640 1 -- start start 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.345+0000 7f6d0b691640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d0410a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.345+0000 7f6d0b691640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d040757a0 con 0x7f6d04107e70 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.345+0000 7f6d0942d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d0410a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.345+0000 7f6d0942d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d0410a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34530/0 (socket says 172.21.15.103:34530) 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.345+0000 7f6d0942d640 1 -- 172.21.15.103:0/3266837194 learned_addr learned my addr 172.21.15.103:0/3266837194 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.345+0000 7f6d0942d640 1 -- 172.21.15.103:0/3266837194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0410a7a0 con 0x7f6d04107e70 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.346+0000 7f6d0942d640 1 --2- 172.21.15.103:0/3266837194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d0410a260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009820 tx=0x7f6d0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f90ca099768a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.346+0000 7f6cfbfff640 1 -- 172.21.15.103:0/3266837194 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d00037070 con 0x7f6d04107e70 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.346+0000 7f6cfbfff640 1 -- 172.21.15.103:0/3266837194 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d000328f0 con 0x7f6d04107e70 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.346+0000 7f6cfbfff640 1 -- 172.21.15.103:0/3266837194 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d00030740 con 0x7f6d04107e70 2024-09-05T01:53:28.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.347+0000 7f6d0b691640 1 -- 172.21.15.103:0/3266837194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 msgr2=0x7f6d0410a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.347+0000 7f6d0b691640 1 --2- 172.21.15.103:0/3266837194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d0410a260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009820 tx=0x7f6d0002a270 comp rx=0 tx=0).stop 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.347+0000 7f6d0b691640 1 -- 172.21.15.103:0/3266837194 shutdown_connections 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.347+0000 7f6d0b691640 1 --2- 172.21.15.103:0/3266837194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d0410a260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.347+0000 7f6d0b691640 1 -- 172.21.15.103:0/3266837194 >> 172.21.15.103:0/3266837194 conn(0x7f6d04101a50 msgr2=0x7f6d04103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.347+0000 7f6d0b691640 1 -- 172.21.15.103:0/3266837194 shutdown_connections 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.347+0000 7f6d0b691640 1 -- 172.21.15.103:0/3266837194 wait complete. 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.348+0000 7f6d0b691640 1 Processor -- start 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.348+0000 7f6d0b691640 1 -- start start 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.348+0000 7f6d0b691640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d041076f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.348+0000 7f6d0b691640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410b820 con 0x7f6d04107e70 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.348+0000 7f6d0942d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d041076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:28.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.348+0000 7f6d0942d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d041076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34544/0 (socket says 172.21.15.103:34544) 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.348+0000 7f6d0942d640 1 -- 172.21.15.103:0/423072397 learned_addr learned my addr 172.21.15.103:0/423072397 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.348+0000 7f6d0942d640 1 -- 172.21.15.103:0/423072397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d04107c30 con 0x7f6d04107e70 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.348+0000 7f6d0942d640 1 --2- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d041076f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d000327b0 tx=0x7f6d000327e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.349+0000 7f6cfa7fc640 1 -- 172.21.15.103:0/423072397 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d00042070 con 0x7f6d04107e70 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.349+0000 7f6cfa7fc640 1 -- 172.21.15.103:0/423072397 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d00030d50 con 0x7f6d04107e70 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.349+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d04105e20 con 0x7f6d04107e70 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.349+0000 7f6cfa7fc640 1 -- 172.21.15.103:0/423072397 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d00037040 con 0x7f6d04107e70 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.349+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d04106360 con 0x7f6d04107e70 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.349+0000 7f6cfa7fc640 1 -- 172.21.15.103:0/423072397 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f6d00045070 con 0x7f6d04107e70 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.350+0000 7f6cfa7fc640 1 --2- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6cf0038f70 0x7f6cf003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.350+0000 7f6cfa7fc640 1 -- 172.21.15.103:0/423072397 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6d0006bf70 con 0x7f6d04107e70 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.350+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d041082d0 con 0x7f6d04107e70 2024-09-05T01:53:28.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.350+0000 7f6d08c2c640 1 --2- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6cf0038f70 0x7f6cf003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.350+0000 7f6d08c2c640 1 --2- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6cf0038f70 0x7f6cf003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40098c0 tx=0x7f6cf4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.355+0000 7f6cfa7fc640 1 -- 172.21.15.103:0/423072397 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d00045360 con 0x7f6d04107e70 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.513+0000 7f6cfa7fc640 1 -- 172.21.15.103:0/423072397 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f6d0003b330 con 0x7f6d04107e70 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.546+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6d04106c40 con 0x7f6cf0038f70 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.590+0000 7f6cfa7fc640 1 -- 172.21.15.103:0/423072397 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6d04106c40 con 0x7f6cf0038f70 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6cf0038f70 msgr2=0x7f6cf003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 --2- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6cf0038f70 0x7f6cf003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40098c0 tx=0x7f6cf4006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 msgr2=0x7f6d041076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 --2- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d041076f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d000327b0 tx=0x7f6d000327e0 comp rx=0 tx=0).stop 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 shutdown_connections 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 --2- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6cf0038f70 0x7f6cf003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 --2- 172.21.15.103:0/423072397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d04107e70 0x7f6d041076f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 >> 172.21.15.103:0/423072397 conn(0x7f6d04101a50 msgr2=0x7f6d04102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:28.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 shutdown_connections 2024-09-05T01:53:28.235 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:27.593+0000 7f6d0b691640 1 -- 172.21.15.103:0/423072397 wait complete. 2024-09-05T01:53:28.235 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial admin user... 2024-09-05T01:53:28.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:28 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:26] ENGINE Bus STARTING 2024-09-05T01:53:28.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:28 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:26] ENGINE Serving on http://172.21.15.103:8765 2024-09-05T01:53:28.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:28 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:26] ENGINE Serving on https://172.21.15.103:7150 2024-09-05T01:53:28.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:28 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:26] ENGINE Bus STARTED 2024-09-05T01:53:28.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:28 smithi103 ceph-mon[30902]: [05/Sep/2024:01:53:26] ENGINE Error in HTTPServer.serve 2024-09-05T01:53:28.332 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-05T01:53:28.332 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T01:53:28.332 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-05T01:53:28.332 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-05T01:53:28.333 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T01:53:28.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:28.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:28 smithi103 ceph-mon[30902]: mgrmap e12: x(active, since 2s) 2024-09-05T01:53:28.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:28.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:29.289 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:29 smithi103 ceph-mon[30902]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:30.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$SxqKLbArq9qI7qZUQ1cuBu.fFAY7JV2t4CVq9O1IzQYKMDiYbffKG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725501209, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-05T01:53:30.444 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.373+0000 7f435eda2640 1 Processor -- start 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.373+0000 7f435eda2640 1 -- start start 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.373+0000 7f435eda2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f4358107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.373+0000 7f435eda2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4358107e20 con 0x7f4358107450 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.374+0000 7f435cb3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f4358107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.374+0000 7f435cb3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f4358107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39330/0 (socket says 172.21.15.103:39330) 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.374+0000 7f435cb3e640 1 -- 172.21.15.103:0/4276378024 learned_addr learned my addr 172.21.15.103:0/4276378024 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.374+0000 7f435cb3e640 1 -- 172.21.15.103:0/4276378024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4358108650 con 0x7f4358107450 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.374+0000 7f435cb3e640 1 --2- 172.21.15.103:0/4276378024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f4358107850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f434c009820 tx=0x7f434c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0896797d7bd7c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f43577fe640 1 -- 172.21.15.103:0/4276378024 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c037070 con 0x7f4358107450 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f43577fe640 1 -- 172.21.15.103:0/4276378024 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c0328f0 con 0x7f4358107450 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f43577fe640 1 -- 172.21.15.103:0/4276378024 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c030740 con 0x7f4358107450 2024-09-05T01:53:30.445 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f435eda2640 1 -- 172.21.15.103:0/4276378024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 msgr2=0x7f4358107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f435eda2640 1 --2- 172.21.15.103:0/4276378024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f4358107850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f434c009820 tx=0x7f434c02a270 comp rx=0 tx=0).stop 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f435eda2640 1 -- 172.21.15.103:0/4276378024 shutdown_connections 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f435eda2640 1 --2- 172.21.15.103:0/4276378024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f4358107850 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f435eda2640 1 -- 172.21.15.103:0/4276378024 >> 172.21.15.103:0/4276378024 conn(0x7f4358102bc0 msgr2=0x7f4358105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f435eda2640 1 -- 172.21.15.103:0/4276378024 shutdown_connections 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.375+0000 7f435eda2640 1 -- 172.21.15.103:0/4276378024 wait complete. 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.376+0000 7f435eda2640 1 Processor -- start 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.376+0000 7f435eda2640 1 -- start start 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.376+0000 7f435eda2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f435819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.376+0000 7f435eda2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f435810ba00 con 0x7f4358107450 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.376+0000 7f435cb3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f435819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.376+0000 7f435cb3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f435819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39344/0 (socket says 172.21.15.103:39344) 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.376+0000 7f435cb3e640 1 -- 172.21.15.103:0/1352610292 learned_addr learned my addr 172.21.15.103:0/1352610292 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:30.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.376+0000 7f435cb3e640 1 -- 172.21.15.103:0/1352610292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435819b870 con 0x7f4358107450 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.377+0000 7f435cb3e640 1 --2- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f435819b330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f434c009940 tx=0x7f434c030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.377+0000 7f4355ffb640 1 -- 172.21.15.103:0/1352610292 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c042070 con 0x7f4358107450 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.377+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435819bb00 con 0x7f4358107450 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.377+0000 7f4355ffb640 1 -- 172.21.15.103:0/1352610292 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c030e30 con 0x7f4358107450 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.377+0000 7f4355ffb640 1 -- 172.21.15.103:0/1352610292 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c037040 con 0x7f4358107450 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.377+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435819e7f0 con 0x7f4358107450 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.378+0000 7f4355ffb640 1 -- 172.21.15.103:0/1352610292 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f434c045070 con 0x7f4358107450 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.378+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f435810bb80 con 0x7f4358107450 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.378+0000 7f4355ffb640 1 --2- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4348039080 0x7f434803b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.378+0000 7f4355ffb640 1 -- 172.21.15.103:0/1352610292 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f434c06c420 con 0x7f4358107450 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.378+0000 7f4357fff640 1 --2- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4348039080 0x7f434803b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.379+0000 7f4357fff640 1 --2- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4348039080 0x7f434803b540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f43440098c0 tx=0x7f4344006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:30.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.383+0000 7f4355ffb640 1 -- 172.21.15.103:0/1352610292 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f434c03add0 con 0x7f4358107450 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.562+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f43581078d0 con 0x7f4348039080 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.790+0000 7f4355ffb640 1 -- 172.21.15.103:0/1352610292 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f43581078d0 con 0x7f4348039080 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.792+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4348039080 msgr2=0x7f434803b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.792+0000 7f435eda2640 1 --2- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4348039080 0x7f434803b540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f43440098c0 tx=0x7f4344006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.792+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 msgr2=0x7f435819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.792+0000 7f435eda2640 1 --2- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f435819b330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f434c009940 tx=0x7f434c030cc0 comp rx=0 tx=0).stop 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.793+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 shutdown_connections 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.793+0000 7f435eda2640 1 --2- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4348039080 0x7f434803b540 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.793+0000 7f435eda2640 1 --2- 172.21.15.103:0/1352610292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4358107450 0x7f435819b330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.793+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 >> 172.21.15.103:0/1352610292 conn(0x7f4358102bc0 msgr2=0x7f4358103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.793+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 shutdown_connections 2024-09-05T01:53:30.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:29.793+0000 7f435eda2640 1 -- 172.21.15.103:0/1352610292 wait complete. 2024-09-05T01:53:30.449 INFO:teuthology.orchestra.run.smithi103.stdout:Fetching dashboard port number... 2024-09-05T01:53:31.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:30 smithi103 ceph-mon[30902]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:31.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:31.600 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 8443 2024-09-05T01:53:31.600 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.826+0000 7f561c8b9640 1 Processor -- start 2024-09-05T01:53:31.600 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.826+0000 7f561c8b9640 1 -- start start 2024-09-05T01:53:31.600 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.826+0000 7f561c8b9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f5618106760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:31.600 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.826+0000 7f561c8b9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5618106d30 con 0x7f5618106360 2024-09-05T01:53:31.600 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.827+0000 7f561659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f5618106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:31.600 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.827+0000 7f561659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f5618106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39352/0 (socket says 172.21.15.103:39352) 2024-09-05T01:53:31.600 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.827+0000 7f561659c640 1 -- 172.21.15.103:0/3195118031 learned_addr learned my addr 172.21.15.103:0/3195118031 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.827+0000 7f561659c640 1 -- 172.21.15.103:0/3195118031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5618107560 con 0x7f5618106360 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.828+0000 7f561659c640 1 --2- 172.21.15.103:0/3195118031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f5618106760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5608009820 tx=0x7f560802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fc650e6aae6da8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.828+0000 7f561559a640 1 -- 172.21.15.103:0/3195118031 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5608037070 con 0x7f5618106360 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.828+0000 7f561559a640 1 -- 172.21.15.103:0/3195118031 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56080328f0 con 0x7f5618106360 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.828+0000 7f561559a640 1 -- 172.21.15.103:0/3195118031 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5608030740 con 0x7f5618106360 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.829+0000 7f561c8b9640 1 -- 172.21.15.103:0/3195118031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 msgr2=0x7f5618106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.829+0000 7f561c8b9640 1 --2- 172.21.15.103:0/3195118031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f5618106760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5608009820 tx=0x7f560802a270 comp rx=0 tx=0).stop 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.829+0000 7f561c8b9640 1 -- 172.21.15.103:0/3195118031 shutdown_connections 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.829+0000 7f561c8b9640 1 --2- 172.21.15.103:0/3195118031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f5618106760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.829+0000 7f561c8b9640 1 -- 172.21.15.103:0/3195118031 >> 172.21.15.103:0/3195118031 conn(0x7f5618101af0 msgr2=0x7f5618103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.829+0000 7f561c8b9640 1 -- 172.21.15.103:0/3195118031 shutdown_connections 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.829+0000 7f561c8b9640 1 -- 172.21.15.103:0/3195118031 wait complete. 2024-09-05T01:53:31.601 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.829+0000 7f561c8b9640 1 Processor -- start 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.830+0000 7f561c8b9640 1 -- start start 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.830+0000 7f561c8b9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f561819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.830+0000 7f561c8b9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f561810a910 con 0x7f5618106360 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.830+0000 7f561659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f561819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.830+0000 7f561659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f561819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39362/0 (socket says 172.21.15.103:39362) 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.830+0000 7f561659c640 1 -- 172.21.15.103:0/2620085936 learned_addr learned my addr 172.21.15.103:0/2620085936 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.830+0000 7f561659c640 1 -- 172.21.15.103:0/2620085936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561819b8f0 con 0x7f5618106360 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.830+0000 7f561659c640 1 --2- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f561819b3b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56080097f0 tx=0x7f5608030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.830+0000 7f560f7fe640 1 -- 172.21.15.103:0/2620085936 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5608042070 con 0x7f5618106360 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.831+0000 7f560f7fe640 1 -- 172.21.15.103:0/2620085936 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5608030e80 con 0x7f5618106360 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.831+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561819bb80 con 0x7f5618106360 2024-09-05T01:53:31.602 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.831+0000 7f560f7fe640 1 -- 172.21.15.103:0/2620085936 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5608037040 con 0x7f5618106360 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.831+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561819e870 con 0x7f5618106360 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.831+0000 7f560f7fe640 1 -- 172.21.15.103:0/2620085936 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f5608045070 con 0x7f5618106360 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.832+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56181067e0 con 0x7f5618106360 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.832+0000 7f560f7fe640 1 --2- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5604039080 0x7f560403b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.832+0000 7f560f7fe640 1 -- 172.21.15.103:0/2620085936 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f560806c1a0 con 0x7f5618106360 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.832+0000 7f5615d9b640 1 --2- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5604039080 0x7f560403b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.832+0000 7f5615d9b640 1 --2- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5604039080 0x7f560403b540 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56000098c0 tx=0x7f5600006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:30.837+0000 7f560f7fe640 1 -- 172.21.15.103:0/2620085936 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5608045520 con 0x7f5618106360 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.005+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f561802d9a0 con 0x7f5618106360 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.005+0000 7f560f7fe640 1 -- 172.21.15.103:0/2620085936 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f560803add0 con 0x7f5618106360 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5604039080 msgr2=0x7f560403b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 --2- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5604039080 0x7f560403b540 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56000098c0 tx=0x7f5600006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:31.603 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 msgr2=0x7f561819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:31.604 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 --2- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f561819b3b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56080097f0 tx=0x7f5608030c70 comp rx=0 tx=0).stop 2024-09-05T01:53:31.604 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 shutdown_connections 2024-09-05T01:53:31.604 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 --2- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5604039080 0x7f560403b540 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:31.604 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 --2- 172.21.15.103:0/2620085936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5618106360 0x7f561819b3b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:31.604 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 >> 172.21.15.103:0/2620085936 conn(0x7f5618101af0 msgr2=0x7f5618102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:31.604 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 shutdown_connections 2024-09-05T01:53:31.604 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:31.008+0000 7f561c8b9640 1 -- 172.21.15.103:0/2620085936 wait complete. 2024-09-05T01:53:31.606 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T01:53:31.606 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-05T01:53:31.613 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T01:53:31.614 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-05T01:53:31.614 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-05T01:53:31.614 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-05T01:53:31.615 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph Dashboard is now available at: 2024-09-05T01:53:31.615 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:31.615 INFO:teuthology.orchestra.run.smithi103.stdout: URL: https://smithi103:8443/ 2024-09-05T01:53:31.615 INFO:teuthology.orchestra.run.smithi103.stdout: User: admin 2024-09-05T01:53:31.615 INFO:teuthology.orchestra.run.smithi103.stdout: Password: bhf1jkcf0n 2024-09-05T01:53:31.615 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:31.616 INFO:teuthology.orchestra.run.smithi103.stdout:Saving cluster configuration to /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config directory 2024-09-05T01:53:32.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:31 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2620085936' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-05T01:53:32.849 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:32 smithi103 ceph-mon[30902]: mgrmap e13: x(active, since 6s) 2024-09-05T01:53:32.850 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:32 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/133809896' entity='client.admin' 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.015+0000 7efdf9b71640 1 Processor -- start 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.015+0000 7efdf9b71640 1 -- start start 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.015+0000 7efdf9b71640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf4107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.015+0000 7efdf9b71640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efdf4107e20 con 0x7efdf4107450 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.015+0000 7efdf37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf4107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.016+0000 7efdf37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf4107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39370/0 (socket says 172.21.15.103:39370) 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.016+0000 7efdf37fe640 1 -- 172.21.15.103:0/3135416126 learned_addr learned my addr 172.21.15.103:0/3135416126 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.016+0000 7efdf37fe640 1 -- 172.21.15.103:0/3135416126 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf4108650 con 0x7efdf4107450 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.016+0000 7efdf37fe640 1 --2- 172.21.15.103:0/3135416126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf4107850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efde4009820 tx=0x7efde402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d3ac942280e9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf27fc640 1 -- 172.21.15.103:0/3135416126 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde4037070 con 0x7efdf4107450 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf27fc640 1 -- 172.21.15.103:0/3135416126 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efde40328f0 con 0x7efdf4107450 2024-09-05T01:53:32.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf27fc640 1 -- 172.21.15.103:0/3135416126 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde4030740 con 0x7efdf4107450 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf9b71640 1 -- 172.21.15.103:0/3135416126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 msgr2=0x7efdf4107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf9b71640 1 --2- 172.21.15.103:0/3135416126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf4107850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efde4009820 tx=0x7efde402a270 comp rx=0 tx=0).stop 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf9b71640 1 -- 172.21.15.103:0/3135416126 shutdown_connections 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf9b71640 1 --2- 172.21.15.103:0/3135416126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf4107850 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf9b71640 1 -- 172.21.15.103:0/3135416126 >> 172.21.15.103:0/3135416126 conn(0x7efdf4102bc0 msgr2=0x7efdf4105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf9b71640 1 -- 172.21.15.103:0/3135416126 shutdown_connections 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.017+0000 7efdf9b71640 1 -- 172.21.15.103:0/3135416126 wait complete. 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.018+0000 7efdf9b71640 1 Processor -- start 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.018+0000 7efdf9b71640 1 -- start start 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.018+0000 7efdf9b71640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf407db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.018+0000 7efdf9b71640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efdf410ba00 con 0x7efdf4107450 2024-09-05T01:53:32.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.018+0000 7efdf37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf407db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.018+0000 7efdf37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf407db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39372/0 (socket says 172.21.15.103:39372) 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.018+0000 7efdf37fe640 1 -- 172.21.15.103:0/133809896 learned_addr learned my addr 172.21.15.103:0/133809896 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.019+0000 7efdf37fe640 1 -- 172.21.15.103:0/133809896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf407e050 con 0x7efdf4107450 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.019+0000 7efdf37fe640 1 --2- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf407db10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efde40097f0 tx=0x7efde4030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.019+0000 7efdf0ff9640 1 -- 172.21.15.103:0/133809896 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde4042070 con 0x7efdf4107450 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.019+0000 7efdf0ff9640 1 -- 172.21.15.103:0/133809896 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efde4030d80 con 0x7efdf4107450 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.019+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf407c240 con 0x7efdf4107450 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.019+0000 7efdf0ff9640 1 -- 172.21.15.103:0/133809896 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde4037040 con 0x7efdf4107450 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.019+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf407c780 con 0x7efdf4107450 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.020+0000 7efdf0ff9640 1 -- 172.21.15.103:0/133809896 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7efde4045070 con 0x7efdf4107450 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.020+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf410bb80 con 0x7efdf4107450 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.020+0000 7efdf0ff9640 1 --2- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efddc039120 0x7efddc03b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.020+0000 7efdf0ff9640 1 -- 172.21.15.103:0/133809896 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efde406c460 con 0x7efdf4107450 2024-09-05T01:53:32.859 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.020+0000 7efdf2ffd640 1 --2- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efddc039120 0x7efddc03b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.021+0000 7efdf2ffd640 1 --2- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efddc039120 0x7efddc03b5e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efde80098c0 tx=0x7efde8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.025+0000 7efdf0ff9640 1 -- 172.21.15.103:0/133809896 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efde403add0 con 0x7efdf4107450 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.280+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7efdf41078d0 con 0x7efdf4107450 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.283+0000 7efdf0ff9640 1 -- 172.21.15.103:0/133809896 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7efde4045520 con 0x7efdf4107450 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efddc039120 msgr2=0x7efddc03b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 --2- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efddc039120 0x7efddc03b5e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efde80098c0 tx=0x7efde8006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 msgr2=0x7efdf407db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 --2- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf407db10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efde40097f0 tx=0x7efde4030c70 comp rx=0 tx=0).stop 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 shutdown_connections 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 --2- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efddc039120 0x7efddc03b5e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 --2- 172.21.15.103:0/133809896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4107450 0x7efdf407db10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 >> 172.21.15.103:0/133809896 conn(0x7efdf4102bc0 msgr2=0x7efdf4103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:32.860 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 shutdown_connections 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-05T01:53:32.286+0000 7efdf9b71640 1 -- 172.21.15.103:0/133809896 wait complete. 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout:Or, if you are only running a single cluster on this host: 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout: ceph telemetry on 2024-09-05T01:53:32.861 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:32.862 INFO:teuthology.orchestra.run.smithi103.stdout:For more information see: 2024-09-05T01:53:32.862 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:32.862 INFO:teuthology.orchestra.run.smithi103.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-05T01:53:32.862 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:32.862 INFO:teuthology.orchestra.run.smithi103.stdout:Bootstrap complete. 2024-09-05T01:53:32.894 INFO:tasks.cephadm:Fetching config... 2024-09-05T01:53:32.895 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:53:32.895 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-05T01:53:32.910 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-05T01:53:32.911 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:53:32.911 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-05T01:53:32.967 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-05T01:53:32.967 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:53:32.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-05T01:53:33.032 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-05T01:53:33.032 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:53:33.032 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-05T01:53:33.087 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-05T01:53:33.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEdzahn1JOPLH1n3QJZdpCt/MyHfXbgCZcNTi4kRsIRlxv67dvKSJE7Y4quAESW2hBsLi5eRKHVGQ1nVSyDJj7AOBtObngHdRnGwblW5jaFdO4cbOcT8Zm3mfZAZHJlab0xQcBO0bzHCJasMNW5VO/2o5xHciRTbOwu71khkogI978EHIX2Ne2UFfPhS8PK+Vpn8gA5XiY7zlxKz/LmFDrDDun3GKC7wq6njDJ7xLnBS2ayZm81QiSmp7yYPx5IOsBGzrLTadJJBkCxqcTlNR+ug9BZIfCQODYLARwdlISUk3d6yrtJ53ziTLH3Mc4x0uGa54I4CMiIV6tAchvDrKNK8eoJWoiPP0ZW5xMdc+0HWU6yt2P04CvyFZLQx/HPmjmBLgJ5bEgopW4Nhvl0DOPuTtKBg/tK3ol2N/uIDvdPWjLWMCvfQpXizbwWM010/PX8RuoEcr9v+1KMk3M+e53aKjyx6qBqcpRaIzdJ7cNjLe6YF3gKomP3xW4Izmnwd8= ceph-695e8e42-6b29-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T01:53:33.162 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEdzahn1JOPLH1n3QJZdpCt/MyHfXbgCZcNTi4kRsIRlxv67dvKSJE7Y4quAESW2hBsLi5eRKHVGQ1nVSyDJj7AOBtObngHdRnGwblW5jaFdO4cbOcT8Zm3mfZAZHJlab0xQcBO0bzHCJasMNW5VO/2o5xHciRTbOwu71khkogI978EHIX2Ne2UFfPhS8PK+Vpn8gA5XiY7zlxKz/LmFDrDDun3GKC7wq6njDJ7xLnBS2ayZm81QiSmp7yYPx5IOsBGzrLTadJJBkCxqcTlNR+ug9BZIfCQODYLARwdlISUk3d6yrtJ53ziTLH3Mc4x0uGa54I4CMiIV6tAchvDrKNK8eoJWoiPP0ZW5xMdc+0HWU6yt2P04CvyFZLQx/HPmjmBLgJ5bEgopW4Nhvl0DOPuTtKBg/tK3ol2N/uIDvdPWjLWMCvfQpXizbwWM010/PX8RuoEcr9v+1KMk3M+e53aKjyx6qBqcpRaIzdJ7cNjLe6YF3gKomP3xW4Izmnwd8= ceph-695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:53:33.173 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEdzahn1JOPLH1n3QJZdpCt/MyHfXbgCZcNTi4kRsIRlxv67dvKSJE7Y4quAESW2hBsLi5eRKHVGQ1nVSyDJj7AOBtObngHdRnGwblW5jaFdO4cbOcT8Zm3mfZAZHJlab0xQcBO0bzHCJasMNW5VO/2o5xHciRTbOwu71khkogI978EHIX2Ne2UFfPhS8PK+Vpn8gA5XiY7zlxKz/LmFDrDDun3GKC7wq6njDJ7xLnBS2ayZm81QiSmp7yYPx5IOsBGzrLTadJJBkCxqcTlNR+ug9BZIfCQODYLARwdlISUk3d6yrtJ53ziTLH3Mc4x0uGa54I4CMiIV6tAchvDrKNK8eoJWoiPP0ZW5xMdc+0HWU6yt2P04CvyFZLQx/HPmjmBLgJ5bEgopW4Nhvl0DOPuTtKBg/tK3ol2N/uIDvdPWjLWMCvfQpXizbwWM010/PX8RuoEcr9v+1KMk3M+e53aKjyx6qBqcpRaIzdJ7cNjLe6YF3gKomP3xW4Izmnwd8= ceph-695e8e42-6b29-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T01:53:33.214 INFO:teuthology.orchestra.run.smithi139.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEdzahn1JOPLH1n3QJZdpCt/MyHfXbgCZcNTi4kRsIRlxv67dvKSJE7Y4quAESW2hBsLi5eRKHVGQ1nVSyDJj7AOBtObngHdRnGwblW5jaFdO4cbOcT8Zm3mfZAZHJlab0xQcBO0bzHCJasMNW5VO/2o5xHciRTbOwu71khkogI978EHIX2Ne2UFfPhS8PK+Vpn8gA5XiY7zlxKz/LmFDrDDun3GKC7wq6njDJ7xLnBS2ayZm81QiSmp7yYPx5IOsBGzrLTadJJBkCxqcTlNR+ug9BZIfCQODYLARwdlISUk3d6yrtJ53ziTLH3Mc4x0uGa54I4CMiIV6tAchvDrKNK8eoJWoiPP0ZW5xMdc+0HWU6yt2P04CvyFZLQx/HPmjmBLgJ5bEgopW4Nhvl0DOPuTtKBg/tK3ol2N/uIDvdPWjLWMCvfQpXizbwWM010/PX8RuoEcr9v+1KMk3M+e53aKjyx6qBqcpRaIzdJ7cNjLe6YF3gKomP3xW4Izmnwd8= ceph-695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:53:33.227 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEdzahn1JOPLH1n3QJZdpCt/MyHfXbgCZcNTi4kRsIRlxv67dvKSJE7Y4quAESW2hBsLi5eRKHVGQ1nVSyDJj7AOBtObngHdRnGwblW5jaFdO4cbOcT8Zm3mfZAZHJlab0xQcBO0bzHCJasMNW5VO/2o5xHciRTbOwu71khkogI978EHIX2Ne2UFfPhS8PK+Vpn8gA5XiY7zlxKz/LmFDrDDun3GKC7wq6njDJ7xLnBS2ayZm81QiSmp7yYPx5IOsBGzrLTadJJBkCxqcTlNR+ug9BZIfCQODYLARwdlISUk3d6yrtJ53ziTLH3Mc4x0uGa54I4CMiIV6tAchvDrKNK8eoJWoiPP0ZW5xMdc+0HWU6yt2P04CvyFZLQx/HPmjmBLgJ5bEgopW4Nhvl0DOPuTtKBg/tK3ol2N/uIDvdPWjLWMCvfQpXizbwWM010/PX8RuoEcr9v+1KMk3M+e53aKjyx6qBqcpRaIzdJ7cNjLe6YF3gKomP3xW4Izmnwd8= ceph-695e8e42-6b29-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T01:53:33.268 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEdzahn1JOPLH1n3QJZdpCt/MyHfXbgCZcNTi4kRsIRlxv67dvKSJE7Y4quAESW2hBsLi5eRKHVGQ1nVSyDJj7AOBtObngHdRnGwblW5jaFdO4cbOcT8Zm3mfZAZHJlab0xQcBO0bzHCJasMNW5VO/2o5xHciRTbOwu71khkogI978EHIX2Ne2UFfPhS8PK+Vpn8gA5XiY7zlxKz/LmFDrDDun3GKC7wq6njDJ7xLnBS2ayZm81QiSmp7yYPx5IOsBGzrLTadJJBkCxqcTlNR+ug9BZIfCQODYLARwdlISUk3d6yrtJ53ziTLH3Mc4x0uGa54I4CMiIV6tAchvDrKNK8eoJWoiPP0ZW5xMdc+0HWU6yt2P04CvyFZLQx/HPmjmBLgJ5bEgopW4Nhvl0DOPuTtKBg/tK3ol2N/uIDvdPWjLWMCvfQpXizbwWM010/PX8RuoEcr9v+1KMk3M+e53aKjyx6qBqcpRaIzdJ7cNjLe6YF3gKomP3xW4Izmnwd8= ceph-695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:53:33.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-05T01:53:33.527 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.039+0000 7f1aed49c640 1 -- 172.21.15.103:0/399705466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 msgr2=0x7f1ae8101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.039+0000 7f1aed49c640 1 --2- 172.21.15.103:0/399705466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 0x7f1ae8101a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009880 tx=0x7f1ad802a5d0 comp rx=0 tx=0).stop 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.039+0000 7f1aed49c640 1 -- 172.21.15.103:0/399705466 shutdown_connections 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.039+0000 7f1aed49c640 1 --2- 172.21.15.103:0/399705466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 0x7f1ae8101a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.039+0000 7f1aed49c640 1 -- 172.21.15.103:0/399705466 >> 172.21.15.103:0/399705466 conn(0x7f1ae80fcdf0 msgr2=0x7f1ae80ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.039+0000 7f1aed49c640 1 -- 172.21.15.103:0/399705466 shutdown_connections 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.039+0000 7f1aed49c640 1 -- 172.21.15.103:0/399705466 wait complete. 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.040+0000 7f1aed49c640 1 Processor -- start 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.040+0000 7f1aed49c640 1 -- start start 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.040+0000 7f1aed49c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 0x7f1ae8196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.040+0000 7f1aed49c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81075e0 con 0x7f1ae8101620 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.040+0000 7f1ae6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 0x7f1ae8196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.040+0000 7f1ae6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 0x7f1ae8196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39384/0 (socket says 172.21.15.103:39384) 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.040+0000 7f1ae6ffd640 1 -- 172.21.15.103:0/3429910882 learned_addr learned my addr 172.21.15.103:0/3429910882 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.040+0000 7f1ae6ffd640 1 -- 172.21.15.103:0/3429910882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae8197220 con 0x7f1ae8101620 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.040+0000 7f1ae6ffd640 1 --2- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 0x7f1ae8196ce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009230 tx=0x7f1ad80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.041+0000 7f1acffff640 1 -- 172.21.15.103:0/3429910882 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad803f070 con 0x7f1ae8101620 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.041+0000 7f1acffff640 1 -- 172.21.15.103:0/3429910882 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ad8003aa0 con 0x7f1ae8101620 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.041+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae81974b0 con 0x7f1ae8101620 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.041+0000 7f1acffff640 1 -- 172.21.15.103:0/3429910882 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad8035030 con 0x7f1ae8101620 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.041+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae819a190 con 0x7f1ae8101620 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.041+0000 7f1acffff640 1 -- 172.21.15.103:0/3429910882 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1ad8042070 con 0x7f1ae8101620 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.042+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae8107d60 con 0x7f1ae8101620 2024-09-05T01:53:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.042+0000 7f1acffff640 1 --2- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1ad00390c0 0x7f1ad003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:34.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.042+0000 7f1acffff640 1 -- 172.21.15.103:0/3429910882 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1ad806cdf0 con 0x7f1ae8101620 2024-09-05T01:53:34.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.042+0000 7f1ae67fc640 1 --2- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1ad00390c0 0x7f1ad003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:34.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.042+0000 7f1ae67fc640 1 --2- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1ad00390c0 0x7f1ad003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0098c0 tx=0x7f1adc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:34.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.046+0000 7f1acffff640 1 -- 172.21.15.103:0/3429910882 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ad802f300 con 0x7f1ae8101620 2024-09-05T01:53:34.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.217+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f1ae802d940 con 0x7f1ae8101620 2024-09-05T01:53:34.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.219+0000 7f1acffff640 1 -- 172.21.15.103:0/3429910882 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f1ad8042360 con 0x7f1ae8101620 2024-09-05T01:53:34.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.221+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1ad00390c0 msgr2=0x7f1ad003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:34.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.221+0000 7f1aed49c640 1 --2- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1ad00390c0 0x7f1ad003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0098c0 tx=0x7f1adc006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:34.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.221+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 msgr2=0x7f1ae8196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:34.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.221+0000 7f1aed49c640 1 --2- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 0x7f1ae8196ce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009230 tx=0x7f1ad80048f0 comp rx=0 tx=0).stop 2024-09-05T01:53:34.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.222+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 shutdown_connections 2024-09-05T01:53:34.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.222+0000 7f1aed49c640 1 --2- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1ad00390c0 0x7f1ad003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:34.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.222+0000 7f1aed49c640 1 --2- 172.21.15.103:0/3429910882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae8101620 0x7f1ae8196ce0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:34.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.222+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 >> 172.21.15.103:0/3429910882 conn(0x7f1ae80fcdf0 msgr2=0x7f1ae80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:34.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.222+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 shutdown_connections 2024-09-05T01:53:34.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:34.222+0000 7f1aed49c640 1 -- 172.21.15.103:0/3429910882 wait complete. 2024-09-05T01:53:34.840 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-05T01:53:34.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-05T01:53:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:53:35.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:35 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3429910882' entity='client.admin' 2024-09-05T01:53:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.728+0000 7f973f09e640 1 -- 172.21.15.103:0/3969976568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 msgr2=0x7f97381044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.728+0000 7f973f09e640 1 --2- 172.21.15.103:0/3969976568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 0x7f97381044b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9728009c20 tx=0x7f972802aa90 comp rx=0 tx=0).stop 2024-09-05T01:53:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.728+0000 7f973f09e640 1 -- 172.21.15.103:0/3969976568 shutdown_connections 2024-09-05T01:53:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.728+0000 7f973f09e640 1 --2- 172.21.15.103:0/3969976568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 0x7f97381044b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.728+0000 7f973f09e640 1 -- 172.21.15.103:0/3969976568 >> 172.21.15.103:0/3969976568 conn(0x7f97380fbca0 msgr2=0x7f97380fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.728+0000 7f973f09e640 1 -- 172.21.15.103:0/3969976568 shutdown_connections 2024-09-05T01:53:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.728+0000 7f973f09e640 1 -- 172.21.15.103:0/3969976568 wait complete. 2024-09-05T01:53:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f973f09e640 1 Processor -- start 2024-09-05T01:53:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f973f09e640 1 -- start start 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f973f09e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 0x7f9738196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f973f09e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9738107580 con 0x7f97381020c0 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f973ce3a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 0x7f9738196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f973ce3a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 0x7f9738196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39398/0 (socket says 172.21.15.103:39398) 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f973ce3a640 1 -- 172.21.15.103:0/1081378868 learned_addr learned my addr 172.21.15.103:0/1081378868 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f973ce3a640 1 -- 172.21.15.103:0/1081378868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97381971c0 con 0x7f97381020c0 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f973ce3a640 1 --2- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 0x7f9738196c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9728006010 tx=0x7f9728030f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.729+0000 7f9735ffb640 1 -- 172.21.15.103:0/1081378868 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9728047070 con 0x7f97381020c0 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.730+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9738197450 con 0x7f97381020c0 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.730+0000 7f9735ffb640 1 -- 172.21.15.103:0/1081378868 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9728003e20 con 0x7f97381020c0 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.730+0000 7f9735ffb640 1 -- 172.21.15.103:0/1081378868 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9728033050 con 0x7f97381020c0 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.730+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973819a140 con 0x7f97381020c0 2024-09-05T01:53:35.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.731+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9738107d00 con 0x7f97381020c0 2024-09-05T01:53:35.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.731+0000 7f9735ffb640 1 -- 172.21.15.103:0/1081378868 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f972804a070 con 0x7f97381020c0 2024-09-05T01:53:35.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.731+0000 7f9735ffb640 1 --2- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9724039060 0x7f972403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:35.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.731+0000 7f9735ffb640 1 -- 172.21.15.103:0/1081378868 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f972803e030 con 0x7f97381020c0 2024-09-05T01:53:35.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.731+0000 7f9737fff640 1 --2- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9724039060 0x7f972403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:35.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.732+0000 7f9737fff640 1 --2- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9724039060 0x7f972403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f97200098c0 tx=0x7f9720006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:35.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.736+0000 7f9735ffb640 1 -- 172.21.15.103:0/1081378868 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9728004040 con 0x7f97381020c0 2024-09-05T01:53:35.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.905+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f973802d8f0 con 0x7f9724039060 2024-09-05T01:53:35.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.909+0000 7f9735ffb640 1 -- 172.21.15.103:0/1081378868 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f973802d8f0 con 0x7f9724039060 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9724039060 msgr2=0x7f972403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 --2- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9724039060 0x7f972403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f97200098c0 tx=0x7f9720006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 msgr2=0x7f9738196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 --2- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 0x7f9738196c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9728006010 tx=0x7f9728030f50 comp rx=0 tx=0).stop 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 shutdown_connections 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 --2- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9724039060 0x7f972403b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 --2- 172.21.15.103:0/1081378868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97381020c0 0x7f9738196c80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 >> 172.21.15.103:0/1081378868 conn(0x7f97380fbca0 msgr2=0x7f97380fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 shutdown_connections 2024-09-05T01:53:35.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:35.911+0000 7f973f09e640 1 -- 172.21.15.103:0/1081378868 wait complete. 2024-09-05T01:53:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-05T01:53:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:36 smithi103 ceph-mon[30902]: Updating smithi103:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:53:36.697 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi139 2024-09-05T01:53:36.697 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:53:36.697 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.conf 2024-09-05T01:53:36.718 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:53:36.718 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:53:36.776 INFO:tasks.cephadm:Adding host smithi139 to orchestrator... 2024-09-05T01:53:36.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch host add smithi139 2024-09-05T01:53:37.017 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.452+0000 7ffbda0a4640 1 -- 172.21.15.103:0/2595359346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 msgr2=0x7ffbd4102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.452+0000 7ffbda0a4640 1 --2- 172.21.15.103:0/2595359346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 0x7ffbd4102b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009880 tx=0x7ffbc402a670 comp rx=0 tx=0).stop 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.452+0000 7ffbda0a4640 1 -- 172.21.15.103:0/2595359346 shutdown_connections 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.452+0000 7ffbda0a4640 1 --2- 172.21.15.103:0/2595359346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 0x7ffbd4102b60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.452+0000 7ffbda0a4640 1 -- 172.21.15.103:0/2595359346 >> 172.21.15.103:0/2595359346 conn(0x7ffbd40fc1a0 msgr2=0x7ffbd40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.452+0000 7ffbda0a4640 1 -- 172.21.15.103:0/2595359346 shutdown_connections 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.452+0000 7ffbda0a4640 1 -- 172.21.15.103:0/2595359346 wait complete. 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.452+0000 7ffbda0a4640 1 Processor -- start 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.453+0000 7ffbda0a4640 1 -- start start 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.453+0000 7ffbda0a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 0x7ffbd41003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:37.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.453+0000 7ffbda0a4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd410bba0 con 0x7ffbd4102780 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.453+0000 7ffbd37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 0x7ffbd41003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.453+0000 7ffbd37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 0x7ffbd41003e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39430/0 (socket says 172.21.15.103:39430) 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.453+0000 7ffbd37fe640 1 -- 172.21.15.103:0/3391745890 learned_addr learned my addr 172.21.15.103:0/3391745890 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.453+0000 7ffbd37fe640 1 -- 172.21.15.103:0/3391745890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd41022d0 con 0x7ffbd4102780 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.453+0000 7ffbd37fe640 1 --2- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 0x7ffbd41003e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009230 tx=0x7ffbc40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.454+0000 7ffbd0ff9640 1 -- 172.21.15.103:0/3391745890 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc403f070 con 0x7ffbd4102780 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.454+0000 7ffbd0ff9640 1 -- 172.21.15.103:0/3391745890 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbc4004d10 con 0x7ffbd4102780 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.454+0000 7ffbd0ff9640 1 -- 172.21.15.103:0/3391745890 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc4035030 con 0x7ffbd4102780 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.454+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd4100920 con 0x7ffbd4102780 2024-09-05T01:53:37.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.454+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd4100d20 con 0x7ffbd4102780 2024-09-05T01:53:37.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.455+0000 7ffbd0ff9640 1 -- 172.21.15.103:0/3391745890 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ffbc4042070 con 0x7ffbd4102780 2024-09-05T01:53:37.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.455+0000 7ffbd0ff9640 1 --2- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffbbc039060 0x7ffbbc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:37.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.455+0000 7ffbd0ff9640 1 -- 172.21.15.103:0/3391745890 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffbc406ce90 con 0x7ffbd4102780 2024-09-05T01:53:37.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.455+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd406a7c0 con 0x7ffbd4102780 2024-09-05T01:53:37.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.455+0000 7ffbd2ffd640 1 --2- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffbbc039060 0x7ffbbc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:37.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.455+0000 7ffbd2ffd640 1 --2- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffbbc039060 0x7ffbbc03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098c0 tx=0x7ffbc8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:37.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.460+0000 7ffbd0ff9640 1 -- 172.21.15.103:0/3391745890 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbc4003b10 con 0x7ffbd4102780 2024-09-05T01:53:37.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:37.628+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}) -- 0x7ffbd4101910 con 0x7ffbbc039060 2024-09-05T01:53:38.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:37 smithi103 ceph-mon[30902]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:53:38.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:37 smithi103 ceph-mon[30902]: Updating smithi103:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T01:53:38.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:38.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:38.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:39.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:38 smithi103 ceph-mon[30902]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:40.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:39 smithi103 ceph-mon[30902]: Deploying cephadm binary to smithi139 2024-09-05T01:53:40.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.766+0000 7ffbd0ff9640 1 -- 172.21.15.103:0/3391745890 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ffbd4101910 con 0x7ffbbc039060 2024-09-05T01:53:40.767 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi139' with addr '172.21.15.139' 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffbbc039060 msgr2=0x7ffbbc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 --2- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffbbc039060 0x7ffbbc03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098c0 tx=0x7ffbc8006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 msgr2=0x7ffbd41003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 --2- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 0x7ffbd41003e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009230 tx=0x7ffbc40048f0 comp rx=0 tx=0).stop 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 shutdown_connections 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 --2- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffbbc039060 0x7ffbbc03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 --2- 172.21.15.103:0/3391745890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4102780 0x7ffbd41003e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 >> 172.21.15.103:0/3391745890 conn(0x7ffbd40fc1a0 msgr2=0x7ffbd4105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 shutdown_connections 2024-09-05T01:53:40.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:40.768+0000 7ffbda0a4640 1 -- 172.21.15.103:0/3391745890 wait complete. 2024-09-05T01:53:41.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T01:53:41.612 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:53:42.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:42.037 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:41 smithi103 ceph-mon[30902]: Added host smithi139 2024-09-05T01:53:42.037 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:42.037 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:42.037 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:42.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.040+0000 7f141bab7640 1 -- 172.21.15.103:0/411475629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 msgr2=0x7f1414107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:42.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.040+0000 7f141bab7640 1 --2- 172.21.15.103:0/411475629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 0x7f1414107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1410009880 tx=0x7f141002a5d0 comp rx=0 tx=0).stop 2024-09-05T01:53:42.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.040+0000 7f141bab7640 1 -- 172.21.15.103:0/411475629 shutdown_connections 2024-09-05T01:53:42.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.040+0000 7f141bab7640 1 --2- 172.21.15.103:0/411475629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 0x7f1414107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:42.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.040+0000 7f141bab7640 1 -- 172.21.15.103:0/411475629 >> 172.21.15.103:0/411475629 conn(0x7f14140fd2b0 msgr2=0x7f14140ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:42.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.040+0000 7f141bab7640 1 -- 172.21.15.103:0/411475629 shutdown_connections 2024-09-05T01:53:42.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.040+0000 7f141bab7640 1 -- 172.21.15.103:0/411475629 wait complete. 2024-09-05T01:53:42.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.041+0000 7f141bab7640 1 Processor -- start 2024-09-05T01:53:42.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.041+0000 7f141bab7640 1 -- start start 2024-09-05T01:53:42.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.041+0000 7f141bab7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 0x7f1414196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.041+0000 7f141bab7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f141410bb20 con 0x7f1414107660 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.041+0000 7f1419853640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 0x7f1414196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.041+0000 7f1419853640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 0x7f1414196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32888/0 (socket says 172.21.15.103:32888) 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.041+0000 7f1419853640 1 -- 172.21.15.103:0/3634223144 learned_addr learned my addr 172.21.15.103:0/3634223144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.041+0000 7f1419853640 1 -- 172.21.15.103:0/3634223144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1414197500 con 0x7f1414107660 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.041+0000 7f1419853640 1 --2- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 0x7f1414196fc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1410009230 tx=0x7f14100048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.042+0000 7f140affd640 1 -- 172.21.15.103:0/3634223144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f141003f070 con 0x7f1414107660 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.042+0000 7f140affd640 1 -- 172.21.15.103:0/3634223144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1410004d10 con 0x7f1414107660 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.042+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14141977f0 con 0x7f1414107660 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.042+0000 7f140affd640 1 -- 172.21.15.103:0/3634223144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1410035030 con 0x7f1414107660 2024-09-05T01:53:42.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.042+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141419b890 con 0x7f1414107660 2024-09-05T01:53:42.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.042+0000 7f140affd640 1 -- 172.21.15.103:0/3634223144 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1410042070 con 0x7f1414107660 2024-09-05T01:53:42.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.043+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1414102d10 con 0x7f1414107660 2024-09-05T01:53:42.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.043+0000 7f140affd640 1 --2- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1400039060 0x7f140003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:42.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.043+0000 7f140affd640 1 -- 172.21.15.103:0/3634223144 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f141006cc70 con 0x7f1414107660 2024-09-05T01:53:42.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.043+0000 7f1419052640 1 --2- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1400039060 0x7f140003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:42.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.043+0000 7f1419052640 1 --2- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1400039060 0x7f140003b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f14040098c0 tx=0x7f1404006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:42.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.047+0000 7f140affd640 1 -- 172.21.15.103:0/3634223144 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1410042360 con 0x7f1414107660 2024-09-05T01:53:42.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.217+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1414107af0 con 0x7f1400039060 2024-09-05T01:53:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.218+0000 7f140affd640 1 -- 172.21.15.103:0/3634223144 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f1414107af0 con 0x7f1400039060 2024-09-05T01:53:42.219 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:42.219 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}] 2024-09-05T01:53:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.220+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1400039060 msgr2=0x7f140003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.220+0000 7f141bab7640 1 --2- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1400039060 0x7f140003b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f14040098c0 tx=0x7f1404006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.221+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 msgr2=0x7f1414196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.221+0000 7f141bab7640 1 --2- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 0x7f1414196fc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1410009230 tx=0x7f14100048f0 comp rx=0 tx=0).stop 2024-09-05T01:53:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.221+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 shutdown_connections 2024-09-05T01:53:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.221+0000 7f141bab7640 1 --2- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1400039060 0x7f140003b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.221+0000 7f141bab7640 1 --2- 172.21.15.103:0/3634223144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1414107660 0x7f1414196fc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.221+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 >> 172.21.15.103:0/3634223144 conn(0x7f14140fd2b0 msgr2=0x7f14140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.221+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 shutdown_connections 2024-09-05T01:53:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:42.221+0000 7f141bab7640 1 -- 172.21.15.103:0/3634223144 wait complete. 2024-09-05T01:53:42.826 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-09-05T01:53:42.826 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:53:42.826 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-09-05T01:53:42.846 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:53:42.847 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:53:42.904 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-09-05T01:53:42.904 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch host add smithi167 2024-09-05T01:53:43.145 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:53:43.360 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:43 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:43.360 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:43 smithi103 ceph-mon[30902]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:53:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.558+0000 7f90a36b3640 1 -- 172.21.15.103:0/3712272274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 msgr2=0x7f909c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.558+0000 7f90a36b3640 1 --2- 172.21.15.103:0/3712272274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 0x7f909c107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9098009880 tx=0x7f909802a5d0 comp rx=0 tx=0).stop 2024-09-05T01:53:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.558+0000 7f90a36b3640 1 -- 172.21.15.103:0/3712272274 shutdown_connections 2024-09-05T01:53:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.558+0000 7f90a36b3640 1 --2- 172.21.15.103:0/3712272274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 0x7f909c107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.558+0000 7f90a36b3640 1 -- 172.21.15.103:0/3712272274 >> 172.21.15.103:0/3712272274 conn(0x7f909c0fd2b0 msgr2=0x7f909c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.558+0000 7f90a36b3640 1 -- 172.21.15.103:0/3712272274 shutdown_connections 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.558+0000 7f90a36b3640 1 -- 172.21.15.103:0/3712272274 wait complete. 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.558+0000 7f90a36b3640 1 Processor -- start 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90a36b3640 1 -- start start 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90a36b3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 0x7f909c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90a36b3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f909c10bb20 con 0x7f909c107660 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90a144f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 0x7f909c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90a144f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 0x7f909c196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32920/0 (socket says 172.21.15.103:32920) 2024-09-05T01:53:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90a144f640 1 -- 172.21.15.103:0/2920148539 learned_addr learned my addr 172.21.15.103:0/2920148539 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90a144f640 1 -- 172.21.15.103:0/2920148539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c197500 con 0x7f909c107660 2024-09-05T01:53:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90a144f640 1 --2- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 0x7f909c196fc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9098009230 tx=0x7f90980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90927fc640 1 -- 172.21.15.103:0/2920148539 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f909803f070 con 0x7f909c107660 2024-09-05T01:53:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909c1977f0 con 0x7f909c107660 2024-09-05T01:53:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90927fc640 1 -- 172.21.15.103:0/2920148539 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9098004d10 con 0x7f909c107660 2024-09-05T01:53:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.559+0000 7f90927fc640 1 -- 172.21.15.103:0/2920148539 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9098035030 con 0x7f909c107660 2024-09-05T01:53:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.560+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909c19b890 con 0x7f909c107660 2024-09-05T01:53:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.560+0000 7f90927fc640 1 -- 172.21.15.103:0/2920148539 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f9098042070 con 0x7f909c107660 2024-09-05T01:53:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.560+0000 7f90927fc640 1 --2- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f90880390c0 0x7f908803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.560+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f909c102d10 con 0x7f909c107660 2024-09-05T01:53:43.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.561+0000 7f90927fc640 1 -- 172.21.15.103:0/2920148539 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f909806cc10 con 0x7f909c107660 2024-09-05T01:53:43.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.561+0000 7f90a0c4e640 1 --2- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f90880390c0 0x7f908803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:43.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.561+0000 7f90a0c4e640 1 --2- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f90880390c0 0x7f908803b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f908c0098c0 tx=0x7f908c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:43.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.565+0000 7f90927fc640 1 -- 172.21.15.103:0/2920148539 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f909802fdc0 con 0x7f909c107660 2024-09-05T01:53:43.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:43.737+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}) -- 0x7f909c198320 con 0x7f90880390c0 2024-09-05T01:53:44.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:44 smithi103 ceph-mon[30902]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:46.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: Deploying cephadm binary to smithi167 2024-09-05T01:53:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:53:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:53:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:53:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-05T01:53:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:46.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:45 smithi103 ceph-mon[30902]: Updating smithi139:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:53:46.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.982+0000 7f90927fc640 1 -- 172.21.15.103:0/2920148539 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f909c198320 con 0x7f90880390c0 2024-09-05T01:53:46.985 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f90880390c0 msgr2=0x7f908803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 --2- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f90880390c0 0x7f908803b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f908c0098c0 tx=0x7f908c006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 msgr2=0x7f909c196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 --2- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 0x7f909c196fc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9098009230 tx=0x7f90980048f0 comp rx=0 tx=0).stop 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 shutdown_connections 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 --2- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f90880390c0 0x7f908803b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 --2- 172.21.15.103:0/2920148539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f909c107660 0x7f909c196fc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 >> 172.21.15.103:0/2920148539 conn(0x7f909c0fd2b0 msgr2=0x7f909c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 shutdown_connections 2024-09-05T01:53:46.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:46.984+0000 7f90a36b3640 1 -- 172.21.15.103:0/2920148539 wait complete. 2024-09-05T01:53:47.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:46 smithi103 ceph-mon[30902]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:53:47.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:46 smithi103 ceph-mon[30902]: Updating smithi139:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T01:53:47.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:47.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:47.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:47.603 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T01:53:47.841 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:53:48.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:47 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:48.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:47 smithi103 ceph-mon[30902]: Added host smithi167 2024-09-05T01:53:48.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:47 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:48.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:47 smithi103 ceph-mon[30902]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:48.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:47 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:48.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:47 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:48.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.279+0000 7f29e3e77640 1 -- 172.21.15.103:0/3437908582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 msgr2=0x7f29dc106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:48.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.279+0000 7f29e3e77640 1 --2- 172.21.15.103:0/3437908582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 0x7f29dc106970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009880 tx=0x7f29d402a5d0 comp rx=0 tx=0).stop 2024-09-05T01:53:48.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.279+0000 7f29e3e77640 1 -- 172.21.15.103:0/3437908582 shutdown_connections 2024-09-05T01:53:48.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.279+0000 7f29e3e77640 1 --2- 172.21.15.103:0/3437908582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 0x7f29dc106970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:48.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.279+0000 7f29e3e77640 1 -- 172.21.15.103:0/3437908582 >> 172.21.15.103:0/3437908582 conn(0x7f29dc0fc1a0 msgr2=0x7f29dc0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:48.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.279+0000 7f29e3e77640 1 -- 172.21.15.103:0/3437908582 shutdown_connections 2024-09-05T01:53:48.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.279+0000 7f29e3e77640 1 -- 172.21.15.103:0/3437908582 wait complete. 2024-09-05T01:53:48.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29e3e77640 1 Processor -- start 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29e3e77640 1 -- start start 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29e3e77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 0x7f29dc197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29e3e77640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc10aa70 con 0x7f29dc106590 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29e1c13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 0x7f29dc197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29e1c13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 0x7f29dc197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32936/0 (socket says 172.21.15.103:32936) 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29e1c13640 1 -- 172.21.15.103:0/955844934 learned_addr learned my addr 172.21.15.103:0/955844934 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29e1c13640 1 -- 172.21.15.103:0/955844934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc198390 con 0x7f29dc106590 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29e1c13640 1 --2- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 0x7f29dc197e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009230 tx=0x7f29d40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.280+0000 7f29daffd640 1 -- 172.21.15.103:0/955844934 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d403f070 con 0x7f29dc106590 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.281+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc198620 con 0x7f29dc106590 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.281+0000 7f29daffd640 1 -- 172.21.15.103:0/955844934 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29d4004d10 con 0x7f29dc106590 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.281+0000 7f29daffd640 1 -- 172.21.15.103:0/955844934 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d4035030 con 0x7f29dc106590 2024-09-05T01:53:48.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.281+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc198a80 con 0x7f29dc106590 2024-09-05T01:53:48.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.282+0000 7f29daffd640 1 -- 172.21.15.103:0/955844934 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f29d4042070 con 0x7f29dc106590 2024-09-05T01:53:48.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.282+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc101c00 con 0x7f29dc106590 2024-09-05T01:53:48.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.282+0000 7f29daffd640 1 --2- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f29c40390c0 0x7f29c403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:48.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.282+0000 7f29daffd640 1 -- 172.21.15.103:0/955844934 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f29d406cc40 con 0x7f29dc106590 2024-09-05T01:53:48.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.282+0000 7f29e1412640 1 --2- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f29c40390c0 0x7f29c403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:48.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.283+0000 7f29e1412640 1 --2- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f29c40390c0 0x7f29c403b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29d00098c0 tx=0x7f29d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:48.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.287+0000 7f29daffd640 1 -- 172.21.15.103:0/955844934 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29d4042360 con 0x7f29dc106590 2024-09-05T01:53:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.458+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29dc1069f0 con 0x7f29c40390c0 2024-09-05T01:53:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.460+0000 7f29daffd640 1 -- 172.21.15.103:0/955844934 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f29dc1069f0 con 0x7f29c40390c0 2024-09-05T01:53:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:53:48.461 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-09-05T01:53:48.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f29c40390c0 msgr2=0x7f29c403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:48.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 --2- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f29c40390c0 0x7f29c403b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29d00098c0 tx=0x7f29d0006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:48.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 msgr2=0x7f29dc197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:48.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 --2- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 0x7f29dc197e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009230 tx=0x7f29d40048f0 comp rx=0 tx=0).stop 2024-09-05T01:53:48.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 shutdown_connections 2024-09-05T01:53:48.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 --2- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f29c40390c0 0x7f29c403b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:48.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 --2- 172.21.15.103:0/955844934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29dc106590 0x7f29dc197e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:48.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 >> 172.21.15.103:0/955844934 conn(0x7f29dc0fc1a0 msgr2=0x7f29dc0fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:48.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 shutdown_connections 2024-09-05T01:53:48.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:48.462+0000 7f29e3e77640 1 -- 172.21.15.103:0/955844934 wait complete. 2024-09-05T01:53:49.064 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-05T01:53:49.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd crush tunables default 2024-09-05T01:53:49.301 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:53:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:49 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:49 smithi103 ceph-mon[30902]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:53:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:49 smithi103 ceph-mon[30902]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:49.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.795+0000 7fac7ea03640 1 -- 172.21.15.103:0/2289166843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 msgr2=0x7fac78106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.795+0000 7fac7ea03640 1 --2- 172.21.15.103:0/2289166843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 0x7fac78106970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fac68009880 tx=0x7fac6802a5d0 comp rx=0 tx=0).stop 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.795+0000 7fac7ea03640 1 -- 172.21.15.103:0/2289166843 shutdown_connections 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.795+0000 7fac7ea03640 1 --2- 172.21.15.103:0/2289166843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 0x7fac78106970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.795+0000 7fac7ea03640 1 -- 172.21.15.103:0/2289166843 >> 172.21.15.103:0/2289166843 conn(0x7fac780fc1a0 msgr2=0x7fac780fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.795+0000 7fac7ea03640 1 -- 172.21.15.103:0/2289166843 shutdown_connections 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.795+0000 7fac7ea03640 1 -- 172.21.15.103:0/2289166843 wait complete. 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.795+0000 7fac7ea03640 1 Processor -- start 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.795+0000 7fac7ea03640 1 -- start start 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac7ea03640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 0x7fac78102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac7ea03640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac7810aa70 con 0x7fac78106590 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac77fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 0x7fac78102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac77fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 0x7fac78102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35810/0 (socket says 172.21.15.103:35810) 2024-09-05T01:53:49.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac77fff640 1 -- 172.21.15.103:0/826196089 learned_addr learned my addr 172.21.15.103:0/826196089 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac77fff640 1 -- 172.21.15.103:0/826196089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac78104040 con 0x7fac78106590 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac77fff640 1 --2- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 0x7fac78102150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac68009230 tx=0x7fac680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac757fa640 1 -- 172.21.15.103:0/826196089 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6803f070 con 0x7fac78106590 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac757fa640 1 -- 172.21.15.103:0/826196089 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac68004d10 con 0x7fac78106590 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac757fa640 1 -- 172.21.15.103:0/826196089 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac68035030 con 0x7fac78106590 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac78102690 con 0x7fac78106590 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.796+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac78102b50 con 0x7fac78106590 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.797+0000 7fac757fa640 1 -- 172.21.15.103:0/826196089 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fac68042070 con 0x7fac78106590 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.797+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac78101c00 con 0x7fac78106590 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.798+0000 7fac757fa640 1 --2- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fac60039060 0x7fac6003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.798+0000 7fac757fa640 1 -- 172.21.15.103:0/826196089 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fac6806cce0 con 0x7fac78106590 2024-09-05T01:53:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.798+0000 7fac777fe640 1 --2- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fac60039060 0x7fac6003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.798+0000 7fac777fe640 1 --2- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fac60039060 0x7fac6003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0098c0 tx=0x7fac6c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.802+0000 7fac757fa640 1 -- 172.21.15.103:0/826196089 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac680372d0 con 0x7fac78106590 2024-09-05T01:53:49.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:49.972+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fac78103680 con 0x7fac78106590 2024-09-05T01:53:50.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.336+0000 7fac757fa640 1 -- 172.21.15.103:0/826196089 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fac68003920 con 0x7fac78106590 2024-09-05T01:53:50.337 INFO:teuthology.orchestra.run.smithi103.stderr:adjusted tunables profile to default 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fac60039060 msgr2=0x7fac6003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 --2- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fac60039060 0x7fac6003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0098c0 tx=0x7fac6c006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 msgr2=0x7fac78102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 --2- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 0x7fac78102150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac68009230 tx=0x7fac680048f0 comp rx=0 tx=0).stop 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 shutdown_connections 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 --2- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fac60039060 0x7fac6003b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 --2- 172.21.15.103:0/826196089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac78106590 0x7fac78102150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 >> 172.21.15.103:0/826196089 conn(0x7fac780fc1a0 msgr2=0x7fac780fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 shutdown_connections 2024-09-05T01:53:50.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:53:50.338+0000 7fac7ea03640 1 -- 172.21.15.103:0/826196089 wait complete. 2024-09-05T01:53:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:50 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/826196089' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T01:53:50.924 INFO:tasks.cephadm:Adding mon.a on smithi103 2024-09-05T01:53:50.924 INFO:tasks.cephadm:Adding mon.b on smithi139 2024-09-05T01:53:50.924 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-09-05T01:53:50.925 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch apply mon '3;smithi103:172.21.15.103=a;smithi139:172.21.15.139=b;smithi167:172.21.15.167=c' 2024-09-05T01:53:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-05T01:53:51.451 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-05T01:53:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:51 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/826196089' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T01:53:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:51 smithi103 ceph-mon[30902]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:53:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:51 smithi103 ceph-mon[30902]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.987+0000 7fd668fda640 1 -- 172.21.15.167:0/1019180431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 msgr2=0x7fd664100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.987+0000 7fd668fda640 1 --2- 172.21.15.167:0/1019180431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 0x7fd664100930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd654009880 tx=0x7fd65402a5d0 comp rx=0 tx=0).stop 2024-09-05T01:53:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.987+0000 7fd668fda640 1 -- 172.21.15.167:0/1019180431 shutdown_connections 2024-09-05T01:53:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.987+0000 7fd668fda640 1 --2- 172.21.15.167:0/1019180431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 0x7fd664100930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:52.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.987+0000 7fd668fda640 1 -- 172.21.15.167:0/1019180431 >> 172.21.15.167:0/1019180431 conn(0x7fd6640fbca0 msgr2=0x7fd6640fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.988+0000 7fd668fda640 1 -- 172.21.15.167:0/1019180431 shutdown_connections 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.988+0000 7fd668fda640 1 -- 172.21.15.167:0/1019180431 wait complete. 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.988+0000 7fd668fda640 1 Processor -- start 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.988+0000 7fd668fda640 1 -- start start 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.988+0000 7fd668fda640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 0x7fd664198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.988+0000 7fd668fda640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641064f0 con 0x7fd664100530 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.989+0000 7fd66259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 0x7fd664198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.989+0000 7fd66259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 0x7fd664198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.167:40628/0 (socket says 172.21.15.167:40628) 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.989+0000 7fd66259c640 1 -- 172.21.15.167:0/874017040 learned_addr learned my addr 172.21.15.167:0/874017040 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.989+0000 7fd66259c640 1 -- 172.21.15.167:0/874017040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd664199430 con 0x7fd664100530 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.989+0000 7fd66259c640 1 --2- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 0x7fd664198ef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd654009230 tx=0x7fd6540048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:52.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.989+0000 7fd65b7fe640 1 -- 172.21.15.167:0/874017040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65403f070 con 0x7fd664100530 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.990+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6641996c0 con 0x7fd664100530 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.990+0000 7fd65b7fe640 1 -- 172.21.15.167:0/874017040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd654003aa0 con 0x7fd664100530 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.990+0000 7fd65b7fe640 1 -- 172.21.15.167:0/874017040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd654035030 con 0x7fd664100530 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.990+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66419c3b0 con 0x7fd664100530 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.990+0000 7fd65b7fe640 1 -- 172.21.15.167:0/874017040 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fd654042070 con 0x7fd664100530 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.991+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd664106c70 con 0x7fd664100530 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.991+0000 7fd65b7fe640 1 --2- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6500390c0 0x7fd65003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.991+0000 7fd65b7fe640 1 -- 172.21.15.167:0/874017040 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd65406cee0 con 0x7fd664100530 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.991+0000 7fd65bfff640 1 --2- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6500390c0 0x7fd65003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.991+0000 7fd65bfff640 1 --2- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6500390c0 0x7fd65003b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0098c0 tx=0x7fd64c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:52.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:51.996+0000 7fd65b7fe640 1 -- 172.21.15.167:0/874017040 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd654039350 con 0x7fd664100530 2024-09-05T01:53:52.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.189+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi139:172.21.15.139=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}) -- 0x7fd66402d830 con 0x7fd6500390c0 2024-09-05T01:53:52.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.195+0000 7fd65b7fe640 1 -- 172.21.15.167:0/874017040 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd66402d830 con 0x7fd6500390c0 2024-09-05T01:53:52.196 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6500390c0 msgr2=0x7fd65003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 --2- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6500390c0 0x7fd65003b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0098c0 tx=0x7fd64c006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 msgr2=0x7fd664198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 --2- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 0x7fd664198ef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd654009230 tx=0x7fd6540048f0 comp rx=0 tx=0).stop 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 shutdown_connections 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 --2- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6500390c0 0x7fd65003b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 --2- 172.21.15.167:0/874017040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664100530 0x7fd664198ef0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 >> 172.21.15.167:0/874017040 conn(0x7fd6640fbca0 msgr2=0x7fd66418ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 shutdown_connections 2024-09-05T01:53:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:52.197+0000 7fd668fda640 1 -- 172.21.15.167:0/874017040 wait complete. 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: Updating smithi167:/etc/ceph/ceph.conf 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: Updating smithi167:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: Updating smithi167:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T01:53:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi139:172.21.15.139=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:52.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi139:172.21.15.139=b;smithi167:172.21.15.167=c;count:3 2024-09-05T01:53:52.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:52.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:52.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:52.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:52.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:53:52.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:53:52.875 DEBUG:teuthology.orchestra.run.smithi139:mon.b> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.b.service 2024-09-05T01:53:52.878 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.c.service 2024-09-05T01:53:52.882 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T01:53:52.883 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T01:53:53.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:53 smithi103 ceph-mon[30902]: Deploying daemon mon.c on smithi167 2024-09-05T01:53:53.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:53 smithi103 ceph-mon[30902]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:54.463 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:53:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.467+0000 7f0034a30640 1 -- 172.21.15.167:0/1712392321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 msgr2=0x7f0030107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.467+0000 7f0034a30640 1 --2- 172.21.15.167:0/1712392321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 0x7f0030107a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0020009880 tx=0x7f002002a5d0 comp rx=0 tx=0).stop 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.467+0000 7f0034a30640 1 -- 172.21.15.167:0/1712392321 shutdown_connections 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.467+0000 7f0034a30640 1 --2- 172.21.15.167:0/1712392321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 0x7f0030107a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.467+0000 7f0034a30640 1 -- 172.21.15.167:0/1712392321 >> 172.21.15.167:0/1712392321 conn(0x7f0030076250 msgr2=0x7f0030076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.467+0000 7f0034a30640 1 -- 172.21.15.167:0/1712392321 shutdown_connections 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.467+0000 7f0034a30640 1 -- 172.21.15.167:0/1712392321 wait complete. 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.468+0000 7f0034a30640 1 Processor -- start 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.468+0000 7f0034a30640 1 -- start start 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.468+0000 7f0034a30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 0x7f0030197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.468+0000 7f0034a30640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f003010bb90 con 0x7f0030107690 2024-09-05T01:53:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.469+0000 7f002e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 0x7f0030197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.469+0000 7f002e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 0x7f0030197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.167:40644/0 (socket says 172.21.15.167:40644) 2024-09-05T01:53:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.469+0000 7f002e59c640 1 -- 172.21.15.167:0/2409148567 learned_addr learned my addr 172.21.15.167:0/2409148567 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:53:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.469+0000 7f002e59c640 1 -- 172.21.15.167:0/2409148567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0030197650 con 0x7f0030107690 2024-09-05T01:53:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.469+0000 7f002e59c640 1 --2- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 0x7f0030197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0020009230 tx=0x7f00200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.470+0000 7f001f7fe640 1 -- 172.21.15.167:0/2409148567 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f002003f070 con 0x7f0030107690 2024-09-05T01:53:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.470+0000 7f001f7fe640 1 -- 172.21.15.167:0/2409148567 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0020004d10 con 0x7f0030107690 2024-09-05T01:53:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.470+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00301978e0 con 0x7f0030107690 2024-09-05T01:53:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.470+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0030197d40 con 0x7f0030107690 2024-09-05T01:53:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.470+0000 7f001f7fe640 1 -- 172.21.15.167:0/2409148567 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0020035030 con 0x7f0030107690 2024-09-05T01:53:55.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.471+0000 7f001f7fe640 1 -- 172.21.15.167:0/2409148567 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f0020042070 con 0x7f0030107690 2024-09-05T01:53:55.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.471+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0030102d30 con 0x7f0030107690 2024-09-05T01:53:55.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.471+0000 7f001f7fe640 1 --2- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0018039060 0x7f001803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:53:55.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.471+0000 7f001f7fe640 1 -- 172.21.15.167:0/2409148567 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f002006ceb0 con 0x7f0030107690 2024-09-05T01:53:55.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.472+0000 7f002dd9b640 1 --2- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0018039060 0x7f001803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:53:55.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.472+0000 7f002dd9b640 1 --2- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0018039060 0x7f001803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f00240098c0 tx=0x7f0024006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:53:55.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.476+0000 7f001f7fe640 1 -- 172.21.15.167:0/2409148567 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0020036260 con 0x7f0030107690 2024-09-05T01:53:55.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.743+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f00301984d0 con 0x7f0030107690 2024-09-05T01:53:55.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.743+0000 7f001f7fe640 1 -- 172.21.15.167:0/2409148567 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f002004ad50 con 0x7f0030107690 2024-09-05T01:53:55.745 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:53:55.745 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","modified":"2024-09-05T01:52:26.169862Z","created":"2024-09-05T01:52:26.169862Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-05T01:53:55.746 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0018039060 msgr2=0x7f001803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 --2- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0018039060 0x7f001803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f00240098c0 tx=0x7f0024006db0 comp rx=0 tx=0).stop 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 msgr2=0x7f0030197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 --2- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 0x7f0030197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0020009230 tx=0x7f00200048f0 comp rx=0 tx=0).stop 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 shutdown_connections 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 --2- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0018039060 0x7f001803b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 --2- 172.21.15.167:0/2409148567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0030107690 0x7f0030197110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 >> 172.21.15.167:0/2409148567 conn(0x7f0030076250 msgr2=0x7f00300fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 shutdown_connections 2024-09-05T01:53:55.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:53:55.746+0000 7f0034a30640 1 -- 172.21.15.167:0/2409148567 wait complete. 2024-09-05T01:53:56.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:56 smithi103 ceph-mon[30902]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:56.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:53:56 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/2409148567' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:53:56.883 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout: e1 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout: btime 2024-09-05T01:52:29:043577+0000 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout: legacy client fscid: -1 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout: No filesystems configured 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-05T01:53:56.884 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: Added host smithi167 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:56.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:56.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:56.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:56.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:53:56.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:56.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/826196089' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T01:53:56.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/826196089' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T01:53:56.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:53:56.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: Updating smithi167:/etc/ceph/ceph.conf 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: Updating smithi167:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:53:57.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: Updating smithi167:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi139:172.21.15.139=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi139:172.21.15.139=b;smithi167:172.21.15.167=c;count:3 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: Deploying daemon mon.c on smithi167 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/2409148567' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:53:57.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 ceph-mon[35122]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-05T01:53:57.138 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 bash[35103]: f4b9c06b2ea33287dfca271879c89221bdc988cb98701b46b024ce2c9f2ff660 2024-09-05T01:53:57.138 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:53:56 smithi167 systemd[1]: Started Ceph mon.c for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:53:57.401 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T01:53:57.402 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T01:53:57.657 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:54:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: mon.a calling monitor election 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: mon.c calling monitor election 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: monmap epoch 2 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: last_changed 2024-09-05T01:53:56.887502+0000 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: created 2024-09-05T01:52:26.169862+0000 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: min_mon_release 19 (squid) 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: election_strategy: 1 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: fsmap 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:54:02.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: mgrmap e13: x(active, since 36s) 2024-09-05T01:54:02.388 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: overall HEALTH_OK 2024-09-05T01:54:02.388 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:02.388 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:02.388 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:02.388 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:02.388 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:02.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:54:02.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: mon.a calling monitor election 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: mon.c calling monitor election 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: monmap epoch 2 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:54:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: last_changed 2024-09-05T01:53:56.887502+0000 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: created 2024-09-05T01:52:26.169862+0000 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: min_mon_release 19 (squid) 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: election_strategy: 1 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: fsmap 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: mgrmap e13: x(active, since 36s) 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: overall HEALTH_OK 2024-09-05T01:54:02.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:02.401 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:02.401 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:02.401 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:02.401 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:02.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.477+0000 7fd682bf3640 1 -- 172.21.15.167:0/815766087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664003530 msgr2=0x7fd6640059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:02.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.477+0000 7fd682bf3640 1 --2- 172.21.15.167:0/815766087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664003530 0x7fd6640059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd67c1016a0 tx=0x7fd67802c490 comp rx=0 tx=0).stop 2024-09-05T01:54:02.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.477+0000 7fd682bf3640 1 -- 172.21.15.167:0/815766087 shutdown_connections 2024-09-05T01:54:02.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.477+0000 7fd682bf3640 1 --2- 172.21.15.167:0/815766087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd664003530 0x7fd6640059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:02.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.477+0000 7fd682bf3640 1 -- 172.21.15.167:0/815766087 >> 172.21.15.167:0/815766087 conn(0x7fd67c0fc220 msgr2=0x7fd67c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:02.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.477+0000 7fd682bf3640 1 -- 172.21.15.167:0/815766087 shutdown_connections 2024-09-05T01:54:02.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.477+0000 7fd682bf3640 1 -- 172.21.15.167:0/815766087 wait complete. 2024-09-05T01:54:02.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.478+0000 7fd682bf3640 1 Processor -- start 2024-09-05T01:54:02.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.478+0000 7fd682bf3640 1 -- start start 2024-09-05T01:54:02.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.478+0000 7fd682bf3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 0x7fd67c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.478+0000 7fd682bf3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd67c1976f0 0x7fd67c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.478+0000 7fd682bf3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c108520 con 0x7fd67c1976f0 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.478+0000 7fd682bf3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c1083a0 con 0x7fd664003530 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd68098f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 0x7fd67c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd68098f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 0x7fd67c1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53570/0 (socket says 172.21.15.167:53570) 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd68098f640 1 -- 172.21.15.167:0/2583355414 learned_addr learned my addr 172.21.15.167:0/2583355414 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd68098f640 1 -- 172.21.15.167:0/2583355414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 msgr2=0x7fd67c1971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd68098f640 1 -- 172.21.15.167:0/2583355414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 msgr2=0x7fd67c1971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd68098f640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 0x7fd67c1971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd673fff640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd67c1976f0 0x7fd67c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd68098f640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 0x7fd67c1971b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:54:02.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd673fff640 1 -- 172.21.15.167:0/2583355414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 msgr2=0x7fd67c1971b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:54:02.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd673fff640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 0x7fd67c1971b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:02.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd673fff640 1 -- 172.21.15.167:0/2583355414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c197d10 con 0x7fd67c1976f0 2024-09-05T01:54:02.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.479+0000 7fd673fff640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd67c1976f0 0x7fd67c19ba80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00b3d0 tx=0x7fd66c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:02.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.480+0000 7fd671ffb640 1 -- 172.21.15.167:0/2583355414 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd66c011630 con 0x7fd67c1976f0 2024-09-05T01:54:02.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.480+0000 7fd671ffb640 1 -- 172.21.15.167:0/2583355414 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd66c011c80 con 0x7fd67c1976f0 2024-09-05T01:54:02.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.480+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c19bfc0 con 0x7fd67c1976f0 2024-09-05T01:54:02.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.480+0000 7fd671ffb640 1 -- 172.21.15.167:0/2583355414 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd66c010d80 con 0x7fd67c1976f0 2024-09-05T01:54:02.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.480+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c19c580 con 0x7fd67c1976f0 2024-09-05T01:54:02.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.481+0000 7fd671ffb640 1 -- 172.21.15.167:0/2583355414 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fd66c018020 con 0x7fd67c1976f0 2024-09-05T01:54:02.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.481+0000 7fd671ffb640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd668038df0 0x7fd66803b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:02.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.481+0000 7fd671ffb640 1 -- 172.21.15.167:0/2583355414 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd66c010510 con 0x7fd67c1976f0 2024-09-05T01:54:02.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.481+0000 7fd68098f640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd668038df0 0x7fd66803b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:02.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.482+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67c19c890 con 0x7fd67c1976f0 2024-09-05T01:54:02.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.482+0000 7fd68098f640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd668038df0 0x7fd66803b2b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd67802ca80 tx=0x7fd67802c9d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:02.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.487+0000 7fd671ffb640 1 -- 172.21.15.167:0/2583355414 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd66c01f020 con 0x7fd67c1976f0 2024-09-05T01:54:02.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.722+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd67c198570 con 0x7fd67c1976f0 2024-09-05T01:54:02.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.723+0000 7fd671ffb640 1 -- 172.21.15.167:0/2583355414 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fd66c018310 con 0x7fd67c1976f0 2024-09-05T01:54:02.724 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:54:02.724 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","modified":"2024-09-05T01:53:56.887502Z","created":"2024-09-05T01:52:26.169862Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T01:54:02.725 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-09-05T01:54:02.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd668038df0 msgr2=0x7fd66803b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:02.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd668038df0 0x7fd66803b2b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd67802ca80 tx=0x7fd67802c9d0 comp rx=0 tx=0).stop 2024-09-05T01:54:02.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd67c1976f0 msgr2=0x7fd67c19ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:02.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd67c1976f0 0x7fd67c19ba80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00b3d0 tx=0x7fd66c00b890 comp rx=0 tx=0).stop 2024-09-05T01:54:02.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 shutdown_connections 2024-09-05T01:54:02.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd67c1976f0 0x7fd67c19ba80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:02.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd668038df0 0x7fd66803b2b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:02.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 --2- 172.21.15.167:0/2583355414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd664003530 0x7fd67c1971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:02.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 >> 172.21.15.167:0/2583355414 conn(0x7fd67c0fc220 msgr2=0x7fd67c06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:02.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 shutdown_connections 2024-09-05T01:54:02.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:02.725+0000 7fd682bf3640 1 -- 172.21.15.167:0/2583355414 wait complete. 2024-09-05T01:54:03.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:02 smithi167 ceph-mon[35122]: Deploying daemon mon.b on smithi139 2024-09-05T01:54:03.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:02 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/2583355414' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:54:03.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:03.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:02 smithi103 ceph-mon[30902]: Deploying daemon mon.b on smithi139 2024-09-05T01:54:03.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:02 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/2583355414' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:54:03.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:04.376 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T01:54:04.376 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T01:54:04.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:03 smithi167 ceph-mon[35122]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:03 smithi103 ceph-mon[30902]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:04.627 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:54:05.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.064+0000 7f403fb37640 1 -- 172.21.15.167:0/4051889298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4038107670 msgr2=0x7f4038107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:05.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.064+0000 7f403fb37640 1 --2- 172.21.15.167:0/4051889298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4038107670 0x7f4038107a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4034009880 tx=0x7f403402a5d0 comp rx=0 tx=0).stop 2024-09-05T01:54:05.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.064+0000 7f403fb37640 1 -- 172.21.15.167:0/4051889298 shutdown_connections 2024-09-05T01:54:05.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.064+0000 7f403fb37640 1 --2- 172.21.15.167:0/4051889298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4038107670 0x7f4038107a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:05.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.064+0000 7f403fb37640 1 -- 172.21.15.167:0/4051889298 >> 172.21.15.167:0/4051889298 conn(0x7f40380fd350 msgr2=0x7f40380ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:05.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.065+0000 7f403fb37640 1 -- 172.21.15.167:0/4051889298 shutdown_connections 2024-09-05T01:54:05.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.065+0000 7f403fb37640 1 -- 172.21.15.167:0/4051889298 wait complete. 2024-09-05T01:54:05.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.065+0000 7f403fb37640 1 Processor -- start 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.065+0000 7f403fb37640 1 -- start start 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.065+0000 7f403fb37640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4038107670 0x7f40381972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.065+0000 7f403fb37640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 0x7f403819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.065+0000 7f403fb37640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f403810bbe0 con 0x7f4038107670 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.065+0000 7f403fb37640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f403810ba60 con 0x7f4038197810 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d0d2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 0x7f403819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d0d2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 0x7f403819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53574/0 (socket says 172.21.15.167:53574) 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d0d2640 1 -- 172.21.15.167:0/2629936781 learned_addr learned my addr 172.21.15.167:0/2629936781 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d0d2640 1 -- 172.21.15.167:0/2629936781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 msgr2=0x7f403819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d0d2640 1 -- 172.21.15.167:0/2629936781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 msgr2=0x7f403819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d0d2640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 0x7f403819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T01:54:05.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d0d2640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 0x7f403819bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d8d3640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4038107670 0x7f40381972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d8d3640 1 -- 172.21.15.167:0/2629936781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 msgr2=0x7f403819bb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d8d3640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 0x7f403819bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d8d3640 1 -- 172.21.15.167:0/2629936781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4038197e30 con 0x7f4038107670 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f403d8d3640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4038107670 0x7f40381972d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4034000c00 tx=0x7f4034004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f402effd640 1 -- 172.21.15.167:0/2629936781 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4034035030 con 0x7f4038107670 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.066+0000 7f402effd640 1 -- 172.21.15.167:0/2629936781 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40340053a0 con 0x7f4038107670 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.067+0000 7f402effd640 1 -- 172.21.15.167:0/2629936781 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4034031830 con 0x7f4038107670 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.067+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403819c090 con 0x7f4038107670 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.067+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403819c5d0 con 0x7f4038107670 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.067+0000 7f402effd640 1 -- 172.21.15.167:0/2629936781 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4034042070 con 0x7f4038107670 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.068+0000 7f402effd640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4024038e00 0x7f402403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:05.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.068+0000 7f402effd640 1 -- 172.21.15.167:0/2629936781 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f403406cdc0 con 0x7f4038107670 2024-09-05T01:54:05.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.068+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4038102db0 con 0x7f4038107670 2024-09-05T01:54:05.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.068+0000 7f403d0d2640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4024038e00 0x7f402403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:05.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.069+0000 7f403d0d2640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4024038e00 0x7f402403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4028009100 tx=0x7f4028008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:05.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.073+0000 7f402effd640 1 -- 172.21.15.167:0/2629936781 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4034039350 con 0x7f4038107670 2024-09-05T01:54:05.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.304+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f403819c880 con 0x7f4038107670 2024-09-05T01:54:05.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.305+0000 7f402effd640 1 -- 172.21.15.167:0/2629936781 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f403402f300 con 0x7f4038107670 2024-09-05T01:54:05.306 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:54:05.306 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","modified":"2024-09-05T01:53:56.887502Z","created":"2024-09-05T01:52:26.169862Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T01:54:05.306 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4024038e00 msgr2=0x7f402403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4024038e00 0x7f402403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4028009100 tx=0x7f4028008040 comp rx=0 tx=0).stop 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4038107670 msgr2=0x7f40381972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4038107670 0x7f40381972d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4034000c00 tx=0x7f4034004e40 comp rx=0 tx=0).stop 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 shutdown_connections 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4024038e00 0x7f402403b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4038197810 0x7f403819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 --2- 172.21.15.167:0/2629936781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4038107670 0x7f40381972d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 >> 172.21.15.167:0/2629936781 conn(0x7f40380fd350 msgr2=0x7f40380fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 shutdown_connections 2024-09-05T01:54:05.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:05.307+0000 7f403fb37640 1 -- 172.21.15.167:0/2629936781 wait complete. 2024-09-05T01:54:06.850 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T01:54:06.851 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T01:54:07.103 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:54:10.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.712+0000 7f4bd3118640 1 -- 172.21.15.167:0/131620375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106650 msgr2=0x7f4bbc005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.712+0000 7f4bd3118640 1 --2- 172.21.15.167:0/131620375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106650 0x7f4bbc005580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0013ce0 tx=0x7f4bc002c350 comp rx=0 tx=0).stop 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.712+0000 7f4bd3118640 1 -- 172.21.15.167:0/131620375 shutdown_connections 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.712+0000 7f4bd3118640 1 --2- 172.21.15.167:0/131620375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106650 0x7f4bbc005580 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.712+0000 7f4bd3118640 1 -- 172.21.15.167:0/131620375 >> 172.21.15.167:0/131620375 conn(0x7f4bcc0fc280 msgr2=0x7f4bcc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.712+0000 7f4bd3118640 1 -- 172.21.15.167:0/131620375 shutdown_connections 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.712+0000 7f4bd3118640 1 -- 172.21.15.167:0/131620375 wait complete. 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.713+0000 7f4bd3118640 1 Processor -- start 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.713+0000 7f4bd3118640 1 -- start start 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.713+0000 7f4bd3118640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 0x7f4bcc199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.713+0000 7f4bd3118640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc199a40 0x7f4bcc19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.713+0000 7f4bd3118640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc19e310 0x7f4bcc19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.713+0000 7f4bd3118640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc108530 con 0x7f4bcc19e310 2024-09-05T01:54:10.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.713+0000 7f4bd3118640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1083b0 con 0x7f4bcc199a40 2024-09-05T01:54:10.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.713+0000 7f4bd3118640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1086b0 con 0x7f4bcc106650 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd0eb4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 0x7f4bcc199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd0eb4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 0x7f4bcc199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45110/0 (socket says 172.21.15.167:45110) 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd0eb4640 1 -- 172.21.15.167:0/2370179375 learned_addr learned my addr 172.21.15.167:0/2370179375 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd16b5640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc19e310 0x7f4bcc19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bcbfff640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc199a40 0x7f4bcc19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd0eb4640 1 -- 172.21.15.167:0/2370179375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 msgr2=0x7f4bcc199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd0eb4640 1 -- 172.21.15.167:0/2370179375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 msgr2=0x7f4bcc199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd0eb4640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 0x7f4bcc199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd0eb4640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 0x7f4bcc199500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bcbfff640 1 -- 172.21.15.167:0/2370179375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc199a40 msgr2=0x7f4bcc19ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bcbfff640 1 -- 172.21.15.167:0/2370179375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc199a40 msgr2=0x7f4bcc19ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T01:54:10.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bcbfff640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc199a40 0x7f4bcc19ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bcbfff640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc199a40 0x7f4bcc19ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd16b5640 1 -- 172.21.15.167:0/2370179375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 msgr2=0x7f4bcc199500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd16b5640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 0x7f4bcc199500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd16b5640 1 -- 172.21.15.167:0/2370179375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc199a40 msgr2=0x7f4bcc19ddd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd16b5640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc199a40 0x7f4bcc19ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd16b5640 1 -- 172.21.15.167:0/2370179375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc1a6080 con 0x7f4bcc19e310 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.714+0000 7f4bd16b5640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc19e310 0x7f4bcc19e7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400bd50 tx=0x7f4bc400be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.715+0000 7f4bc9ffb640 1 -- 172.21.15.167:0/2370179375 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4019070 con 0x7f4bcc19e310 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.715+0000 7f4bc9ffb640 1 -- 172.21.15.167:0/2370179375 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bc40148f0 con 0x7f4bcc19e310 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.715+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc1a62b0 con 0x7f4bcc19e310 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.715+0000 7f4bc9ffb640 1 -- 172.21.15.167:0/2370179375 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4012910 con 0x7f4bcc19e310 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.715+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc1a67c0 con 0x7f4bcc19e310 2024-09-05T01:54:10.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.716+0000 7f4bc9ffb640 1 -- 172.21.15.167:0/2370179375 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4bc401c070 con 0x7f4bcc19e310 2024-09-05T01:54:10.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.716+0000 7f4bc9ffb640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb00394f0 0x7f4bb003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:10.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.716+0000 7f4bc9ffb640 1 -- 172.21.15.167:0/2370179375 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4bc404bd00 con 0x7f4bcc19e310 2024-09-05T01:54:10.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.716+0000 7f4bd0eb4640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb00394f0 0x7f4bb003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:10.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.716+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bcc109620 con 0x7f4bcc19e310 2024-09-05T01:54:10.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.717+0000 7f4bd0eb4640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb00394f0 0x7f4bb003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0005ab0 tx=0x7f4bc002e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:10.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.721+0000 7f4bc9ffb640 1 -- 172.21.15.167:0/2370179375 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bc400e0a0 con 0x7f4bcc19e310 2024-09-05T01:54:10.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.968+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4bcc02d7c0 con 0x7f4bcc19e310 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: mon.a calling monitor election 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: mon.c calling monitor election 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:10.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: monmap epoch 3 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: last_changed 2024-09-05T01:54:05.639622+0000 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: created 2024-09-05T01:52:26.169862+0000 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: min_mon_release 19 (squid) 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: election_strategy: 1 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-09-05T01:54:10.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: fsmap 2024-09-05T01:54:10.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:54:10.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: mgrmap e13: x(active, since 45s) 2024-09-05T01:54:10.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: overall HEALTH_OK 2024-09-05T01:54:10.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:10.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:10.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:10.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:10.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.969+0000 7f4bc9ffb640 1 -- 172.21.15.167:0/2370179375 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f4bc4007290 con 0x7f4bcc19e310 2024-09-05T01:54:10.972 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:54:10.973 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","modified":"2024-09-05T01:54:05.639622Z","created":"2024-09-05T01:52:26.169862Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T01:54:10.973 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb00394f0 msgr2=0x7f4bb003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb00394f0 0x7f4bb003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0005ab0 tx=0x7f4bc002e040 comp rx=0 tx=0).stop 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc19e310 msgr2=0x7f4bcc19e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc19e310 0x7f4bcc19e7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400bd50 tx=0x7f4bc400be40 comp rx=0 tx=0).stop 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 shutdown_connections 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc19e310 0x7f4bcc19e7c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc199a40 0x7f4bcc19ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb00394f0 0x7f4bb003b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 --2- 172.21.15.167:0/2370179375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc106650 0x7f4bcc199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 >> 172.21.15.167:0/2370179375 conn(0x7f4bcc0fc280 msgr2=0x7f4bcc06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.971+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 shutdown_connections 2024-09-05T01:54:10.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:10.972+0000 7f4bd3118640 1 -- 172.21.15.167:0/2370179375 wait complete. 2024-09-05T01:54:11.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:54:11.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: mon.a calling monitor election 2024-09-05T01:54:11.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: mon.c calling monitor election 2024-09-05T01:54:11.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:11.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:54:11.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:11.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:11.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:11.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: monmap epoch 3 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: last_changed 2024-09-05T01:54:05.639622+0000 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: created 2024-09-05T01:52:26.169862+0000 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: min_mon_release 19 (squid) 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: election_strategy: 1 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-05T01:54:11.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-09-05T01:54:11.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-09-05T01:54:11.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: fsmap 2024-09-05T01:54:11.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:54:11.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: mgrmap e13: x(active, since 45s) 2024-09-05T01:54:11.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: overall HEALTH_OK 2024-09-05T01:54:11.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:11.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:11.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:11.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:11.681 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-05T01:54:11.681 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph config generate-minimal-conf 2024-09-05T01:54:11.934 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:54:12.045 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:12.045 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:11 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/2370179375' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:54:12.045 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:11 smithi103 ceph-mon[30902]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:12.045 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:12.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:12.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:11 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/2370179375' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:54:12.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:11 smithi167 ceph-mon[35122]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:12.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:12.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.352+0000 7f55dd022640 1 -- 172.21.15.103:0/2140000945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d8106590 msgr2=0x7f55d8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.352+0000 7f55dd022640 1 --2- 172.21.15.103:0/2140000945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d8106590 0x7f55d8106970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009880 tx=0x7f55c802a5d0 comp rx=0 tx=0).stop 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.352+0000 7f55dd022640 1 -- 172.21.15.103:0/2140000945 shutdown_connections 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.352+0000 7f55dd022640 1 --2- 172.21.15.103:0/2140000945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d8106590 0x7f55d8106970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.352+0000 7f55dd022640 1 -- 172.21.15.103:0/2140000945 >> 172.21.15.103:0/2140000945 conn(0x7f55d80fc1a0 msgr2=0x7f55d80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.352+0000 7f55dd022640 1 -- 172.21.15.103:0/2140000945 shutdown_connections 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.352+0000 7f55dd022640 1 -- 172.21.15.103:0/2140000945 wait complete. 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.353+0000 7f55dd022640 1 Processor -- start 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.353+0000 7f55dd022640 1 -- start start 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55dd022640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d8106590 0x7f55d80fc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55dd022640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d80fcb90 0x7f55d80fcff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55dd022640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55d81a34e0 0x7f55d81a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55dd022640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55d810aa20 con 0x7f55d80fcb90 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55dd022640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f55d810a8a0 con 0x7f55d81a34e0 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55dd022640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55d810aba0 con 0x7f55d8106590 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d80fcb90 0x7f55d80fcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d80fcb90 0x7f55d80fcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56886/0 (socket says 172.21.15.103:56886) 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d659c640 1 -- 172.21.15.103:0/4142023028 learned_addr learned my addr 172.21.15.103:0/4142023028 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d6d9d640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d8106590 0x7f55d80fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d659c640 1 -- 172.21.15.103:0/4142023028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d8106590 msgr2=0x7f55d80fc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d659c640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d8106590 0x7f55d80fc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d659c640 1 -- 172.21.15.103:0/4142023028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55d81a34e0 msgr2=0x7f55d81a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d659c640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55d81a34e0 0x7f55d81a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d659c640 1 -- 172.21.15.103:0/4142023028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d81a5fa0 con 0x7f55d80fcb90 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55d659c640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d80fcb90 0x7f55d80fcff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55c000b520 tx=0x7f55c000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.354+0000 7f55cffff640 1 -- 172.21.15.103:0/4142023028 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0004230 con 0x7f55d80fcb90 2024-09-05T01:54:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.355+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d81a6290 con 0x7f55d80fcb90 2024-09-05T01:54:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.355+0000 7f55cffff640 1 -- 172.21.15.103:0/4142023028 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55c00043d0 con 0x7f55d80fcb90 2024-09-05T01:54:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.355+0000 7f55cffff640 1 -- 172.21.15.103:0/4142023028 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0003cf0 con 0x7f55d80fcb90 2024-09-05T01:54:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.355+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81a6820 con 0x7f55d80fcb90 2024-09-05T01:54:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.356+0000 7f55cffff640 1 -- 172.21.15.103:0/4142023028 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f55c000e070 con 0x7f55d80fcb90 2024-09-05T01:54:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.356+0000 7f55cffff640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f55c40394f0 0x7f55c403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.356+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d81069f0 con 0x7f55d80fcb90 2024-09-05T01:54:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.356+0000 7f55cffff640 1 -- 172.21.15.103:0/4142023028 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f55c004bbf0 con 0x7f55d80fcb90 2024-09-05T01:54:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.356+0000 7f55d6d9d640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f55c40394f0 0x7f55c403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.356+0000 7f55d6d9d640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f55c40394f0 0x7f55c403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f55c8005ab0 tx=0x7f55c8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.361+0000 7f55cffff640 1 -- 172.21.15.103:0/4142023028 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55c0014020 con 0x7f55d80fcb90 2024-09-05T01:54:12.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.536+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f55d81a6a80 con 0x7f55d80fcb90 2024-09-05T01:54:12.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.536+0000 7f55cffff640 1 -- 172.21.15.103:0/4142023028 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f55c0020350 con 0x7f55d80fcb90 2024-09-05T01:54:12.538 INFO:teuthology.orchestra.run.smithi103.stdout:# minimal ceph.conf for 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:54:12.538 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2024-09-05T01:54:12.538 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:54:12.538 INFO:teuthology.orchestra.run.smithi103.stdout: mon_host = [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-09-05T01:54:12.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f55c40394f0 msgr2=0x7f55c403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:12.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f55c40394f0 0x7f55c403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f55c8005ab0 tx=0x7f55c8005a20 comp rx=0 tx=0).stop 2024-09-05T01:54:12.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d80fcb90 msgr2=0x7f55d80fcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:12.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d80fcb90 0x7f55d80fcff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55c000b520 tx=0x7f55c000b9e0 comp rx=0 tx=0).stop 2024-09-05T01:54:12.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 shutdown_connections 2024-09-05T01:54:12.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55d81a34e0 0x7f55d81a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:12.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55d80fcb90 0x7f55d80fcff0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:12.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f55c40394f0 0x7f55c403b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:12.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 --2- 172.21.15.103:0/4142023028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d8106590 0x7f55d80fc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:12.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 >> 172.21.15.103:0/4142023028 conn(0x7f55d80fc1a0 msgr2=0x7f55d818fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:12.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 shutdown_connections 2024-09-05T01:54:12.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:12.540+0000 7f55dd022640 1 -- 172.21.15.103:0/4142023028 wait complete. 2024-09-05T01:54:12.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:12 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:13.418 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-05T01:54:13.418 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:54:13.418 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T01:54:13.445 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:54:13.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:54:13.511 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:54:13.511 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T01:54:13.539 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:54:13.540 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:54:13.605 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:54:13.606 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T01:54:13.634 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:54:13.634 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:54:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: mon.b calling monitor election 2024-09-05T01:54:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: Updating smithi167:/etc/ceph/ceph.conf 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: mon.b calling monitor election 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: mon.c calling monitor election 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: mon.a calling monitor election 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: monmap epoch 3 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: last_changed 2024-09-05T01:54:05.639622+0000 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: created 2024-09-05T01:52:26.169862+0000 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: min_mon_release 19 (squid) 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: election_strategy: 1 2024-09-05T01:54:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: fsmap 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: mgrmap e13: x(active, since 47s) 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: overall HEALTH_OK 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: Updating smithi103:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: Updating smithi167:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: Updating smithi139:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:13.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:13 smithi103 ceph-mon[30902]: Reconfiguring daemon mon.a on smithi103 2024-09-05T01:54:13.701 INFO:tasks.cephadm:Adding mgr.x on smithi103 2024-09-05T01:54:13.701 INFO:tasks.cephadm:Adding mgr.y on smithi139 2024-09-05T01:54:13.701 INFO:tasks.cephadm:Adding mgr.z on smithi167 2024-09-05T01:54:13.701 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch apply mgr '3;smithi103=x;smithi139=y;smithi167=z' 2024-09-05T01:54:13.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: mon.b calling monitor election 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: Updating smithi167:/etc/ceph/ceph.conf 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: mon.b calling monitor election 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: mon.c calling monitor election 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: mon.a calling monitor election 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: monmap epoch 3 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: last_changed 2024-09-05T01:54:05.639622+0000 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: created 2024-09-05T01:52:26.169862+0000 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: min_mon_release 19 (squid) 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: election_strategy: 1 2024-09-05T01:54:13.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: fsmap 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: mgrmap e13: x(active, since 47s) 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: overall HEALTH_OK 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: Updating smithi103:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: Updating smithi167:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: Updating smithi139:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:13.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:54:13.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:13.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:13 smithi167 ceph-mon[35122]: Reconfiguring daemon mon.a on smithi103 2024-09-05T01:54:13.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: mon.b calling monitor election 2024-09-05T01:54:13.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-05T01:54:13.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-05T01:54:13.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: Updating smithi167:/etc/ceph/ceph.conf 2024-09-05T01:54:13.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: mon.b calling monitor election 2024-09-05T01:54:13.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: mon.c calling monitor election 2024-09-05T01:54:13.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: mon.a calling monitor election 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: monmap epoch 3 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: last_changed 2024-09-05T01:54:05.639622+0000 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: created 2024-09-05T01:52:26.169862+0000 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: min_mon_release 19 (squid) 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: election_strategy: 1 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: fsmap 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:54:13.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: mgrmap e13: x(active, since 47s) 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: overall HEALTH_OK 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: Updating smithi103:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: Updating smithi167:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: Updating smithi139:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.899 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:13.899 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T01:54:13.899 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:13.899 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:54:13.899 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:13.899 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:13 smithi139 ceph-mon[34262]: Reconfiguring daemon mon.a on smithi103 2024-09-05T01:54:13.979 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:54:14.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.420+0000 7f703cc1b640 1 -- 172.21.15.167:0/66438713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70380ff9a0 msgr2=0x7f70380ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:14.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.420+0000 7f703cc1b640 1 --2- 172.21.15.167:0/66438713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70380ff9a0 0x7f70380ffda0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7028009880 tx=0x7f702802a5d0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.420+0000 7f703cc1b640 1 -- 172.21.15.167:0/66438713 shutdown_connections 2024-09-05T01:54:14.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.420+0000 7f703cc1b640 1 --2- 172.21.15.167:0/66438713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70380ff9a0 0x7f70380ffda0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.420+0000 7f703cc1b640 1 -- 172.21.15.167:0/66438713 >> 172.21.15.167:0/66438713 conn(0x7f70380fb510 msgr2=0x7f70380fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.420+0000 7f703cc1b640 1 -- 172.21.15.167:0/66438713 shutdown_connections 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.420+0000 7f703cc1b640 1 -- 172.21.15.167:0/66438713 wait complete. 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.421+0000 7f703cc1b640 1 Processor -- start 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.421+0000 7f703cc1b640 1 -- start start 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.421+0000 7f703cc1b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70380ff9a0 0x7f7038196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.421+0000 7f703cc1b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038197440 0x7f703819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.421+0000 7f703cc1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70381978c0 0x7f703819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f703cc1b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70381076a0 con 0x7f7038197440 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f703cc1b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7038107520 con 0x7f70381978c0 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f703cc1b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7038107820 con 0x7f70380ff9a0 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038197440 0x7f703819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038197440 0x7f703819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.167:60312/0 (socket says 172.21.15.167:60312) 2024-09-05T01:54:14.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702ffff640 1 -- 172.21.15.167:0/734741012 learned_addr learned my addr 172.21.15.167:0/734741012 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f703659c640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70380ff9a0 0x7f7038196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f7036d9d640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70381978c0 0x7f703819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702ffff640 1 -- 172.21.15.167:0/734741012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70380ff9a0 msgr2=0x7f7038196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702ffff640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70380ff9a0 0x7f7038196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702ffff640 1 -- 172.21.15.167:0/734741012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70381978c0 msgr2=0x7f703819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702ffff640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70381978c0 0x7f703819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702ffff640 1 -- 172.21.15.167:0/734741012 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703819f300 con 0x7f7038197440 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702ffff640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038197440 0x7f703819c4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f702000b520 tx=0x7f702000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.422+0000 7f702f7fe640 1 -- 172.21.15.167:0/734741012 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020004230 con 0x7f7038197440 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.423+0000 7f702f7fe640 1 -- 172.21.15.167:0/734741012 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70200043d0 con 0x7f7038197440 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.423+0000 7f702f7fe640 1 -- 172.21.15.167:0/734741012 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020003cf0 con 0x7f7038197440 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.423+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703819f5f0 con 0x7f7038197440 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.423+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703819fc00 con 0x7f7038197440 2024-09-05T01:54:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.424+0000 7f702f7fe640 1 -- 172.21.15.167:0/734741012 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f702000e070 con 0x7f7038197440 2024-09-05T01:54:14.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.424+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7038190460 con 0x7f7038197440 2024-09-05T01:54:14.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.424+0000 7f702f7fe640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240394f0 0x7f702403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:14.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.424+0000 7f702f7fe640 1 -- 172.21.15.167:0/734741012 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f702004b960 con 0x7f7038197440 2024-09-05T01:54:14.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.424+0000 7f703659c640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240394f0 0x7f702403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:14.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.425+0000 7f703659c640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240394f0 0x7f702403b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c00 tx=0x7f70280314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:14.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.429+0000 7f702f7fe640 1 -- 172.21.15.167:0/734741012 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7020014020 con 0x7f7038197440 2024-09-05T01:54:14.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.613+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi139=y;smithi167=z", "target": ["mon-mgr", ""]}) -- 0x7f70380626c0 con 0x7f70240394f0 2024-09-05T01:54:14.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.618+0000 7f702f7fe640 1 -- 172.21.15.167:0/734741012 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f70380626c0 con 0x7f70240394f0 2024-09-05T01:54:14.619 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-09-05T01:54:14.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240394f0 msgr2=0x7f702403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240394f0 0x7f702403b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c00 tx=0x7f70280314a0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038197440 msgr2=0x7f703819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038197440 0x7f703819c4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f702000b520 tx=0x7f702000b9e0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 shutdown_connections 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70381978c0 0x7f703819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038197440 0x7f703819c4b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240394f0 0x7f702403b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 --2- 172.21.15.167:0/734741012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70380ff9a0 0x7f7038196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 >> 172.21.15.167:0/734741012 conn(0x7f70380fb510 msgr2=0x7f70380fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 shutdown_connections 2024-09-05T01:54:14.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:54:14.620+0000 7f703cc1b640 1 -- 172.21.15.167:0/734741012 wait complete. 2024-09-05T01:54:14.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:14.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:14.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:14.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:14.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:54:14.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:14.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:14.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:14.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:14.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:14.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:54:14.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:14.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:54:14.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:14.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:14.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:14.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:54:14.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:15.245 DEBUG:teuthology.orchestra.run.smithi139:mgr.y> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.y.service 2024-09-05T01:54:15.248 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.z.service 2024-09-05T01:54:15.251 INFO:tasks.cephadm:Deploying OSDs... 2024-09-05T01:54:15.251 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T01:54:15.251 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T01:54:15.268 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T01:54:15.268 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-09-05T01:54:15.324 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T01:54:15.324 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:15.325 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-09-05T01:54:15.325 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:15.325 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:15.325 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-05 01:53:33.728243936 +0000 2024-09-05T01:54:15.325 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-05 01:49:33.980255782 +0000 2024-09-05T01:54:15.325 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-05 01:49:33.980255782 +0000 2024-09-05T01:54:15.325 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-05 01:49:33.980255782 +0000 2024-09-05T01:54:15.325 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T01:54:15.389 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-05T01:54:15.390 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-05T01:54:15.390 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000220264 s, 2.3 MB/s 2024-09-05T01:54:15.392 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T01:54:15.449 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-09-05T01:54:15.505 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T01:54:15.505 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:15.505 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-05T01:54:15.505 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:15.505 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:15.506 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-05 01:53:33.754243442 +0000 2024-09-05T01:54:15.506 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-05 01:49:34.301249698 +0000 2024-09-05T01:54:15.506 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-05 01:49:34.301249698 +0000 2024-09-05T01:54:15.506 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-05 01:49:34.301249698 +0000 2024-09-05T01:54:15.506 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T01:54:15.569 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-05T01:54:15.569 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-05T01:54:15.570 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000208235 s, 2.5 MB/s 2024-09-05T01:54:15.571 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T01:54:15.627 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-09-05T01:54:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:15 smithi103 ceph-mon[30902]: Reconfiguring mon.b (monmap changed)... 2024-09-05T01:54:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:15 smithi103 ceph-mon[30902]: Reconfiguring daemon mon.b on smithi139 2024-09-05T01:54:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:15 smithi103 ceph-mon[30902]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi139=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:54:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:15 smithi103 ceph-mon[30902]: Saving service mgr spec with placement smithi103=x;smithi139=y;smithi167=z;count:3 2024-09-05T01:54:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:15.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:15 smithi103 ceph-mon[30902]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:15.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T01:54:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:15.663 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-05T01:54:15.664 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:15.664 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:15.664 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-05 01:53:33.780242949 +0000 2024-09-05T01:54:15.664 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-05 01:49:34.633243404 +0000 2024-09-05T01:54:15.664 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-05 01:49:34.633243404 +0000 2024-09-05T01:54:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-05 01:49:34.633243404 +0000 2024-09-05T01:54:15.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T01:54:15.726 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-05T01:54:15.726 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-05T01:54:15.726 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000181321 s, 2.8 MB/s 2024-09-05T01:54:15.727 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T01:54:15.783 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-09-05T01:54:15.838 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T01:54:15.838 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:15.838 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-05T01:54:15.838 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:15.839 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:15.839 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-05 01:53:33.807242437 +0000 2024-09-05T01:54:15.839 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-05 01:49:34.963237149 +0000 2024-09-05T01:54:15.839 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-05 01:49:34.963237149 +0000 2024-09-05T01:54:15.839 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-05 01:49:34.963237149 +0000 2024-09-05T01:54:15.839 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T01:54:15.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:15 smithi167 ceph-mon[35122]: Reconfiguring mon.b (monmap changed)... 2024-09-05T01:54:15.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:15 smithi167 ceph-mon[35122]: Reconfiguring daemon mon.b on smithi139 2024-09-05T01:54:15.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:15 smithi167 ceph-mon[35122]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi139=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:54:15.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:15 smithi167 ceph-mon[35122]: Saving service mgr spec with placement smithi103=x;smithi139=y;smithi167=z;count:3 2024-09-05T01:54:15.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:15.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:15 smithi167 ceph-mon[35122]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:15.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:15.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:15 smithi139 ceph-mon[34262]: Reconfiguring mon.b (monmap changed)... 2024-09-05T01:54:15.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:15 smithi139 ceph-mon[34262]: Reconfiguring daemon mon.b on smithi139 2024-09-05T01:54:15.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:15 smithi139 ceph-mon[34262]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi139=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:54:15.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:15 smithi139 ceph-mon[34262]: Saving service mgr spec with placement smithi103=x;smithi139=y;smithi167=z;count:3 2024-09-05T01:54:15.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:15.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:15 smithi139 ceph-mon[34262]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:15.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:15.901 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-05T01:54:15.901 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-05T01:54:15.901 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000201328 s, 2.5 MB/s 2024-09-05T01:54:15.902 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T01:54:15.959 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-05T01:54:15.960 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T01:54:15.977 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T01:54:15.977 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-09-05T01:54:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T01:54:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:16.032 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-05T01:54:16.032 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:16.032 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:16.032 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-05 01:54:05.062905001 +0000 2024-09-05T01:54:16.032 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-05 01:49:34.193654761 +0000 2024-09-05T01:54:16.033 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-05 01:49:34.193654761 +0000 2024-09-05T01:54:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-05 01:49:34.193654761 +0000 2024-09-05T01:54:16.033 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T01:54:16.095 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-05T01:54:16.095 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-05T01:54:16.095 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000268685 s, 1.9 MB/s 2024-09-05T01:54:16.096 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T01:54:16.154 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-09-05T01:54:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T01:54:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:16.209 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-05T01:54:16.209 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:16.209 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:16.209 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-05 01:54:05.063904980 +0000 2024-09-05T01:54:16.210 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-05 01:49:34.535647447 +0000 2024-09-05T01:54:16.210 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-05 01:49:34.535647447 +0000 2024-09-05T01:54:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-05 01:49:34.535647447 +0000 2024-09-05T01:54:16.210 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T01:54:16.273 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-05T01:54:16.273 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-05T01:54:16.273 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000197734 s, 2.6 MB/s 2024-09-05T01:54:16.274 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T01:54:16.331 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-09-05T01:54:16.386 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T01:54:16.386 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:16.386 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-05T01:54:16.386 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:16.387 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:16.387 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-05 01:54:05.063904980 +0000 2024-09-05T01:54:16.387 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-05 01:49:34.858640539 +0000 2024-09-05T01:54:16.387 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-05 01:49:34.858640539 +0000 2024-09-05T01:54:16.387 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-05 01:49:34.858640539 +0000 2024-09-05T01:54:16.387 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T01:54:16.450 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-05T01:54:16.450 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-05T01:54:16.450 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000217257 s, 2.4 MB/s 2024-09-05T01:54:16.451 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T01:54:16.509 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-09-05T01:54:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T01:54:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:16.565 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-05T01:54:16.565 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:16.565 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:16.565 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-05 01:54:05.064904959 +0000 2024-09-05T01:54:16.565 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-05 01:49:35.185633545 +0000 2024-09-05T01:54:16.565 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-05 01:49:35.185633545 +0000 2024-09-05T01:54:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-05 01:49:35.185633545 +0000 2024-09-05T01:54:16.565 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T01:54:16.628 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-05T01:54:16.628 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-05T01:54:16.628 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000234488 s, 2.2 MB/s 2024-09-05T01:54:16.630 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T01:54:16.686 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-05T01:54:16.686 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T01:54:16.704 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T01:54:16.704 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-09-05T01:54:16.760 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T01:54:16.760 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:16.760 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-05T01:54:16.761 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:16.761 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:16.761 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-05 01:53:55.907601888 +0000 2024-09-05T01:54:16.761 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-05 01:49:35.607245184 +0000 2024-09-05T01:54:16.761 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-05 01:49:35.607245184 +0000 2024-09-05T01:54:16.761 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-09-05 01:49:35.607245184 +0000 2024-09-05T01:54:16.761 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T01:54:16.826 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-05T01:54:16.827 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-05T01:54:16.827 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000273328 s, 1.9 MB/s 2024-09-05T01:54:16.828 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T01:54:16.884 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-09-05T01:54:16.940 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T01:54:16.940 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:16.941 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-05T01:54:16.941 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:16.941 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:16.941 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-05 01:53:55.907601888 +0000 2024-09-05T01:54:16.941 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-05 01:49:35.927239891 +0000 2024-09-05T01:54:16.941 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-05 01:49:35.927239891 +0000 2024-09-05T01:54:16.941 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-09-05 01:49:35.927239891 +0000 2024-09-05T01:54:16.941 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T01:54:17.004 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-05T01:54:17.004 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-05T01:54:17.004 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000201781 s, 2.5 MB/s 2024-09-05T01:54:17.006 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T01:54:17.062 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-09-05T01:54:17.118 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T01:54:17.118 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:17.118 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-05T01:54:17.118 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:17.118 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:17.118 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-05 01:53:55.907601888 +0000 2024-09-05T01:54:17.118 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-05 01:49:36.265234301 +0000 2024-09-05T01:54:17.118 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-05 01:49:36.265234301 +0000 2024-09-05T01:54:17.118 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-09-05 01:49:36.265234301 +0000 2024-09-05T01:54:17.119 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T01:54:17.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:17.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:17.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:16 smithi167 ceph-mon[35122]: Reconfiguring mon.c (monmap changed)... 2024-09-05T01:54:17.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:17.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:54:17.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:17.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:16 smithi167 ceph-mon[35122]: Reconfiguring daemon mon.c on smithi167 2024-09-05T01:54:17.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:17.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:17.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:16 smithi139 ceph-mon[34262]: Reconfiguring mon.c (monmap changed)... 2024-09-05T01:54:17.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:17.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:54:17.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:17.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:16 smithi139 ceph-mon[34262]: Reconfiguring daemon mon.c on smithi167 2024-09-05T01:54:17.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:17.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:17.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:16 smithi103 ceph-mon[30902]: Reconfiguring mon.c (monmap changed)... 2024-09-05T01:54:17.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:54:17.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:54:17.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:17.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:16 smithi103 ceph-mon[30902]: Reconfiguring daemon mon.c on smithi167 2024-09-05T01:54:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-05T01:54:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-05T01:54:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000214813 s, 2.4 MB/s 2024-09-05T01:54:17.160 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T01:54:17.217 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-09-05T01:54:17.273 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T01:54:17.273 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:54:17.273 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-05T01:54:17.273 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:54:17.273 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:54:17.273 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-05 01:53:55.908601872 +0000 2024-09-05T01:54:17.273 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-05 01:49:36.595228843 +0000 2024-09-05T01:54:17.273 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-05 01:49:36.595228843 +0000 2024-09-05T01:54:17.273 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-09-05 01:49:36.595228843 +0000 2024-09-05T01:54:17.273 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T01:54:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-05T01:54:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-05T01:54:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000183701 s, 2.8 MB/s 2024-09-05T01:54:17.339 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T01:54:17.394 INFO:tasks.cephadm:Deploying osd.0 on smithi103 with /dev/vg_nvme/lv_4... 2024-09-05T01:54:17.395 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T01:54:17.640 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:54:18.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:18.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:18.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:18.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:18.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:18.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:18.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:18.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:54:18.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:54:18.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:54:18.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:54:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:18.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:54:18.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:18.912 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:18 smithi103 ceph-mon[30902]: Deploying daemon mgr.z on smithi167 2024-09-05T01:54:19.039 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:18 smithi167 ceph-mon[35122]: Deploying daemon mgr.z on smithi167 2024-09-05T01:54:19.040 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 systemd[1]: Starting Ceph mgr.z for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:54:19.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:18 smithi139 ceph-mon[34262]: Deploying daemon mgr.z on smithi167 2024-09-05T01:54:19.339 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 podman[36916]: 2024-09-05 01:54:19.152263888 +0000 UTC m=+0.015969382 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:54:19.339 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 podman[36916]: 2024-09-05 01:54:19.254907455 +0000 UTC m=+0.118612943 container create 88fae74ca532704254d2f39b6406c796d7ba6544fe141c3ff4a4d5f6935a562d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-05T01:54:19.761 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 podman[36916]: 2024-09-05 01:54:19.424145416 +0000 UTC m=+0.287850902 container init 88fae74ca532704254d2f39b6406c796d7ba6544fe141c3ff4a4d5f6935a562d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-05T01:54:19.761 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 podman[36916]: 2024-09-05 01:54:19.426831627 +0000 UTC m=+0.290537123 container start 88fae74ca532704254d2f39b6406c796d7ba6544fe141c3ff4a4d5f6935a562d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-05T01:54:19.761 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-mgr[36935]: -- 172.21.15.167:0/1173577954 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b61ff4d0e0 con 0x55b61ff2b000 2024-09-05T01:54:19.761 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 bash[36916]: 88fae74ca532704254d2f39b6406c796d7ba6544fe141c3ff4a4d5f6935a562d 2024-09-05T01:54:19.762 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 systemd[1]: Started Ceph mgr.z for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:54:19.762 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:19.548+0000 7f7460b26040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:54:19.762 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:19.612+0000 7f7460b26040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:54:19.762 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:19.760+0000 7f7460b26040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:54:19.996 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:54:20.019 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:19 smithi103 ceph-mon[30902]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:20.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:20.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:20.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:20.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:54:20.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:54:20.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:54:20.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:20.021 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-09-05T01:54:20.118 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:19.830+0000 7f7460b26040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:54:20.118 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:19.977+0000 7f7460b26040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:54:20.118 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:20 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:20.117+0000 7f7460b26040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:54:20.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-mon[35122]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:20.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:20.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:20.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:20.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:54:20.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:54:20.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:54:20.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:20.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:19 smithi139 ceph-mon[34262]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:20.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:20.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:20.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:20.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:54:20.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:54:20.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:54:20.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:20.262 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.678+0000 7fb127512640 1 -- 172.21.15.103:0/3774888767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 msgr2=0x7fb120103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.678+0000 7fb127512640 1 --2- 172.21.15.103:0/3774888767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 0x7fb120103280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb110009900 tx=0x7fb11002a440 comp rx=0 tx=0).stop 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.678+0000 7fb127512640 1 -- 172.21.15.103:0/3774888767 shutdown_connections 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.678+0000 7fb127512640 1 --2- 172.21.15.103:0/3774888767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1201037c0 0x7fb12010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.678+0000 7fb127512640 1 --2- 172.21.15.103:0/3774888767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 0x7fb120103280 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.678+0000 7fb127512640 1 --2- 172.21.15.103:0/3774888767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb120101c00 0x7fb120102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.678+0000 7fb127512640 1 -- 172.21.15.103:0/3774888767 >> 172.21.15.103:0/3774888767 conn(0x7fb1200fd390 msgr2=0x7fb1200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.678+0000 7fb127512640 1 -- 172.21.15.103:0/3774888767 shutdown_connections 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.678+0000 7fb127512640 1 -- 172.21.15.103:0/3774888767 wait complete. 2024-09-05T01:54:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb127512640 1 Processor -- start 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb127512640 1 -- start start 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb127512640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb120101c00 0x7fb120075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb127512640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 0x7fb1200745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb127512640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1201037c0 0x7fb120074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb127512640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb12010d850 con 0x7fb120102e00 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb127512640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb12010d6d0 con 0x7fb1201037c0 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb127512640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb12010d9d0 con 0x7fb120101c00 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb124aad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 0x7fb1200745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb1252ae640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb120101c00 0x7fb120075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb124aad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 0x7fb1200745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33166/0 (socket says 172.21.15.103:33166) 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.679+0000 7fb124aad640 1 -- 172.21.15.103:0/1514763853 learned_addr learned my addr 172.21.15.103:0/1514763853 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:54:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb125aaf640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1201037c0 0x7fb120074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb124aad640 1 -- 172.21.15.103:0/1514763853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb120101c00 msgr2=0x7fb120075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb124aad640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb120101c00 0x7fb120075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb124aad640 1 -- 172.21.15.103:0/1514763853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1201037c0 msgr2=0x7fb120074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb124aad640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1201037c0 0x7fb120074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb124aad640 1 -- 172.21.15.103:0/1514763853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1200753e0 con 0x7fb120102e00 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb1252ae640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb120101c00 0x7fb120075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb124aad640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 0x7fb1200745e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb110009230 tx=0x7fb110004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb1167fc640 1 -- 172.21.15.103:0/1514763853 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11003f070 con 0x7fb120102e00 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1201a20e0 con 0x7fb120102e00 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb1167fc640 1 -- 172.21.15.103:0/1514763853 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb110003aa0 con 0x7fb120102e00 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb1167fc640 1 -- 172.21.15.103:0/1514763853 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110035030 con 0x7fb120102e00 2024-09-05T01:54:20.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.680+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1201a2660 con 0x7fb120102e00 2024-09-05T01:54:20.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.681+0000 7fb1167fc640 1 -- 172.21.15.103:0/1514763853 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fb110042070 con 0x7fb120102e00 2024-09-05T01:54:20.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.681+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1201090f0 con 0x7fb120102e00 2024-09-05T01:54:20.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.681+0000 7fb1167fc640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb10c039290 0x7fb10c03b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:20.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.681+0000 7fb1167fc640 1 -- 172.21.15.103:0/1514763853 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb11006d150 con 0x7fb120102e00 2024-09-05T01:54:20.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.681+0000 7fb1252ae640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb10c039290 0x7fb10c03b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:20.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.682+0000 7fb1252ae640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb10c039290 0x7fb10c03b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb120102c60 tx=0x7fb11c0090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:20.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.686+0000 7fb1167fc640 1 -- 172.21.15.103:0/1514763853 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb110003cc0 con 0x7fb120102e00 2024-09-05T01:54:20.792 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:20 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:20.430+0000 7f7460b26040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:54:20.792 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:20 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:20.492+0000 7f7460b26040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:54:20.792 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:20 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:20.676+0000 7f7460b26040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:54:20.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:20.862+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb120062630 con 0x7fb10c039290 2024-09-05T01:54:21.058 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:20 smithi139 ceph-mon[34262]: Deploying daemon mgr.y on smithi139 2024-09-05T01:54:21.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:20 smithi167 ceph-mon[35122]: Deploying daemon mgr.y on smithi139 2024-09-05T01:54:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:20 smithi103 ceph-mon[30902]: Deploying daemon mgr.y on smithi139 2024-09-05T01:54:21.646 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:21 smithi139 systemd[1]: Starting Ceph mgr.y for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:54:21.885 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:21 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:21.559+0000 7f7460b26040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:54:21.885 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:21 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:21.625+0000 7f7460b26040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:54:21.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:21 smithi167 ceph-mon[35122]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:54:21.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:21 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:54:21.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:21 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:54:21.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:21 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:21.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:21 smithi167 ceph-mon[35122]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:21.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:21 smithi103 ceph-mon[30902]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:54:21.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:21 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:54:21.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:21 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:54:21.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:21 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:21.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:21 smithi103 ceph-mon[30902]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:21.988 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:21 smithi139 ceph-mon[34262]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:54:21.988 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:21 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:54:21.988 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:21 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:54:21.988 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:21 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:21.988 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:21 smithi139 ceph-mon[34262]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:21.988 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:21 smithi139 podman[35613]: 2024-09-05 01:54:21.616485239 +0000 UTC m=+0.015850680 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:54:21.988 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:21 smithi139 podman[35613]: 2024-09-05 01:54:21.727193586 +0000 UTC m=+0.126559026 container create a84e3496d58952c07f482ffd6bdff8fbf7ed2a79db78e771e642780ba8ea1380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-05T01:54:21.989 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:21 smithi139 podman[35613]: 2024-09-05 01:54:21.904548132 +0000 UTC m=+0.303913579 container init a84e3496d58952c07f482ffd6bdff8fbf7ed2a79db78e771e642780ba8ea1380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-05T01:54:21.989 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:21 smithi139 podman[35613]: 2024-09-05 01:54:21.907487322 +0000 UTC m=+0.306852759 container start a84e3496d58952c07f482ffd6bdff8fbf7ed2a79db78e771e642780ba8ea1380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-05T01:54:21.989 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:21 smithi139 ceph-mgr[35632]: -- 172.21.15.139:0/1950319721 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555fca2c70e0 con 0x555fca2a5400 2024-09-05T01:54:21.989 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:21 smithi139 bash[35613]: a84e3496d58952c07f482ffd6bdff8fbf7ed2a79db78e771e642780ba8ea1380 2024-09-05T01:54:22.249 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:21 smithi139 systemd[1]: Started Ceph mgr.y for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:54:22.249 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:22.027+0000 7fa40f06c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:54:22.249 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:22.091+0000 7fa40f06c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:54:22.249 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:22.248+0000 7fa40f06c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:54:22.630 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:22.317+0000 7fa40f06c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:54:22.631 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:22.476+0000 7fa40f06c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:54:22.631 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:22.629+0000 7fa40f06c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:54:22.635 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:22.366+0000 7f7460b26040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:54:22.635 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:22.494+0000 7f7460b26040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:54:22.948 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:22.655+0000 7f7460b26040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:54:22.949 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:22.717+0000 7f7460b26040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:54:22.949 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:22.878+0000 7f7460b26040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:54:22.949 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:22.947+0000 7f7460b26040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:54:23.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:23.211 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:22.954+0000 7fa40f06c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:54:23.211 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:23 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:23.017+0000 7fa40f06c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:54:23.211 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.211 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.211 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.212 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.212 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:23.251 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:23.009+0000 7f7460b26040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:54:23.251 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:23.174+0000 7f7460b26040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:54:23.252 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:23.250+0000 7f7460b26040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:54:23.252 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.252 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.252 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.252 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:23.252 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:23.527 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:23.320+0000 7f7460b26040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:54:23.527 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:23.406+0000 7f7460b26040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:54:23.527 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:23.466+0000 7f7460b26040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:54:23.528 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:23.526+0000 7f7460b26040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:54:23.645 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:23 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:23.209+0000 7fa40f06c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:54:24.385 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z[36931]: 2024-09-05T01:54:24.318+0000 7f7460b26040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:54:24.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1168346506' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fe0c785-c54d-4779-8255-8d88e39aa32d"}]: dispatch 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1168346506' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fe0c785-c54d-4779-8255-8d88e39aa32d"}]': finished 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:24.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:54:24.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:54:24.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:24.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:24 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2609691942' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:54:24.395 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:24.103+0000 7fa40f06c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:54:24.395 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:24.170+0000 7fa40f06c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:54:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1168346506' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fe0c785-c54d-4779-8255-8d88e39aa32d"}]: dispatch 2024-09-05T01:54:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1168346506' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fe0c785-c54d-4779-8255-8d88e39aa32d"}]': finished 2024-09-05T01:54:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T01:54:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:24.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:24.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:54:24.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:54:24.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:24.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2609691942' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:54:24.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:24.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1168346506' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fe0c785-c54d-4779-8255-8d88e39aa32d"}]: dispatch 2024-09-05T01:54:24.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1168346506' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fe0c785-c54d-4779-8255-8d88e39aa32d"}]': finished 2024-09-05T01:54:24.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T01:54:24.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:24.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:24 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2609691942' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:54:25.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:25.083+0000 7fb1167fc640 1 -- 172.21.15.103:0/1514763853 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7fb110032000 con 0x7fb120102e00 2024-09-05T01:54:25.138 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-mon[34262]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T01:54:25.138 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-mon[34262]: Reconfiguring daemon mgr.x on smithi103 2024-09-05T01:54:25.138 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-mon[34262]: Standby manager daemon z started 2024-09-05T01:54:25.138 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-mon[34262]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T01:54:25.139 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-mon[34262]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:54:25.139 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-mon[34262]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T01:54:25.139 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-mon[34262]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:54:25.139 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:25.139 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:24.876+0000 7fa40f06c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:54:25.140 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:24.989+0000 7fa40f06c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:54:25.140 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.136+0000 7fa40f06c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:54:25.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:25 smithi167 ceph-mon[35122]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T01:54:25.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:25 smithi167 ceph-mon[35122]: Reconfiguring daemon mgr.x on smithi103 2024-09-05T01:54:25.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:25 smithi167 ceph-mon[35122]: Standby manager daemon z started 2024-09-05T01:54:25.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:25 smithi167 ceph-mon[35122]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T01:54:25.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:25 smithi167 ceph-mon[35122]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:54:25.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:25 smithi167 ceph-mon[35122]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T01:54:25.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:25 smithi167 ceph-mon[35122]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:54:25.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:25.395 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.197+0000 7fa40f06c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:54:25.395 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.343+0000 7fa40f06c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:54:25.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:25 smithi103 ceph-mon[30902]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T01:54:25.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:25 smithi103 ceph-mon[30902]: Reconfiguring daemon mgr.x on smithi103 2024-09-05T01:54:25.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:25 smithi103 ceph-mon[30902]: Standby manager daemon z started 2024-09-05T01:54:25.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:25 smithi103 ceph-mon[30902]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T01:54:25.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:25 smithi103 ceph-mon[30902]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:54:25.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:25 smithi103 ceph-mon[30902]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T01:54:25.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:25 smithi103 ceph-mon[30902]: from='mgr.? 172.21.15.167:0/1592357712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:54:25.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:25.693 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.412+0000 7fa40f06c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:54:25.694 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.472+0000 7fa40f06c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:54:25.694 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.628+0000 7fa40f06c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:54:25.694 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.692+0000 7fa40f06c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:54:25.959 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.758+0000 7fa40f06c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:54:25.960 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.840+0000 7fa40f06c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:54:25.960 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.899+0000 7fa40f06c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:54:25.960 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:25.958+0000 7fa40f06c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:54:26.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:26 smithi167 ceph-mon[35122]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:26.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:26 smithi167 ceph-mon[35122]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-05T01:54:26.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T01:54:26.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:26 smithi139 ceph-mon[34262]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:26.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:26 smithi139 ceph-mon[34262]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-05T01:54:26.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T01:54:26.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:26 smithi103 ceph-mon[30902]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:26.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:26 smithi103 ceph-mon[30902]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-05T01:54:26.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T01:54:27.080 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 01:54:26 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y[35628]: 2024-09-05T01:54:26.736+0000 7fa40f06c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:54:27.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:27.090+0000 7fb1167fc640 1 -- 172.21.15.103:0/1514763853 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fb11003b460 con 0x7fb120102e00 2024-09-05T01:54:27.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:27 smithi167 ceph-mon[35122]: Standby manager daemon y started 2024-09-05T01:54:27.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:27 smithi167 ceph-mon[35122]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T01:54:27.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:27 smithi167 ceph-mon[35122]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:54:27.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:27 smithi167 ceph-mon[35122]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T01:54:27.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:27 smithi167 ceph-mon[35122]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:54:27.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:27 smithi139 ceph-mon[34262]: Standby manager daemon y started 2024-09-05T01:54:27.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:27 smithi139 ceph-mon[34262]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T01:54:27.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:27 smithi139 ceph-mon[34262]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:54:27.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:27 smithi139 ceph-mon[34262]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T01:54:27.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:27 smithi139 ceph-mon[34262]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:54:27.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:27 smithi103 ceph-mon[30902]: Standby manager daemon y started 2024-09-05T01:54:27.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:27 smithi103 ceph-mon[30902]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T01:54:27.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:27 smithi103 ceph-mon[30902]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:54:27.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:27 smithi103 ceph-mon[30902]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T01:54:27.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:27 smithi103 ceph-mon[30902]: from='mgr.? 172.21.15.139:0/4037902900' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:54:28.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:28 smithi103 ceph-mon[30902]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:28.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:28 smithi103 ceph-mon[30902]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-05T01:54:28.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T01:54:28.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:28 smithi167 ceph-mon[35122]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:28.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:28 smithi167 ceph-mon[35122]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-05T01:54:28.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T01:54:28.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:28 smithi139 ceph-mon[34262]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:28.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:28 smithi139 ceph-mon[34262]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-05T01:54:28.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T01:54:30.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:30 smithi103 ceph-mon[30902]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:30.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:30 smithi167 ceph-mon[35122]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:30.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:30 smithi139 ceph-mon[34262]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:31.393 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:31 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:31.393 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:31 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:31.393 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:31 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:31.394 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:31 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:31.394 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:31 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:31.394 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:31 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:31.394 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:31 smithi103 ceph-mon[30902]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:31 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:31 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:31 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:31.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:31 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:31.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:31 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:31.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:31 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:31.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:31 smithi167 ceph-mon[35122]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:31.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:31 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:31.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:31 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:31 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:31 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:31 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:31 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:31 smithi139 ceph-mon[34262]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:32.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:32 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T01:54:32.901 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:32 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:33.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:32 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T01:54:33.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:32 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:33.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:32 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T01:54:33.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:32 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:34.062 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:33 smithi103 ceph-mon[30902]: Deploying daemon osd.0 on smithi103 2024-09-05T01:54:34.063 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:33 smithi103 ceph-mon[30902]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:34.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:33 smithi167 ceph-mon[35122]: Deploying daemon osd.0 on smithi103 2024-09-05T01:54:34.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:33 smithi167 ceph-mon[35122]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:34.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:33 smithi139 ceph-mon[34262]: Deploying daemon osd.0 on smithi103 2024-09-05T01:54:34.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:33 smithi139 ceph-mon[34262]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:36.376 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:36 smithi103 ceph-mon[30902]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:36.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:36 smithi167 ceph-mon[35122]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:36.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:36 smithi139 ceph-mon[34262]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:38.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:38 smithi103 ceph-mon[30902]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:38.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:38 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:38.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:38 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:38.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:38 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:38.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:38 smithi167 ceph-mon[35122]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:38.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:38 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:38.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:38 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:38.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:38 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:38.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:38 smithi139 ceph-mon[34262]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:38.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:38 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:38.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:38 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:38.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:38 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:40.374 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:40 smithi103 ceph-mon[30902]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:40.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:40 smithi167 ceph-mon[35122]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:40.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:40 smithi139 ceph-mon[34262]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:41.759 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:41.759 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:41.760 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:41.760 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:41.760 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:41.760 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:41 smithi103 ceph-mon[30902]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:41.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:41 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:41.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:41 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:41.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:41 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:41.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:41 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:41.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:41 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:41.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:41 smithi167 ceph-mon[35122]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:41.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:41 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:41.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:41 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:41.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:41 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:41.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:41 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:41.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:41 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:41.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:41 smithi139 ceph-mon[34262]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:42.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.045+0000 7fb1167fc640 1 -- 172.21.15.103:0/1514763853 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb120062630 con 0x7fb10c039290 2024-09-05T01:54:42.047 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 0 on host 'smithi103' 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb10c039290 msgr2=0x7fb10c03b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb10c039290 0x7fb10c03b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb120102c60 tx=0x7fb11c0090e0 comp rx=0 tx=0).stop 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 msgr2=0x7fb1200745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 0x7fb1200745e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb110009230 tx=0x7fb110004430 comp rx=0 tx=0).stop 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 shutdown_connections 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1201037c0 0x7fb120074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb120102e00 0x7fb1200745e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb10c039290 0x7fb10c03b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 --2- 172.21.15.103:0/1514763853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb120101c00 0x7fb120075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 >> 172.21.15.103:0/1514763853 conn(0x7fb1200fd390 msgr2=0x7fb1201062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 shutdown_connections 2024-09-05T01:54:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:54:42.047+0000 7fb127512640 1 -- 172.21.15.103:0/1514763853 wait complete. 2024-09-05T01:54:42.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:42 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:42.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:42 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:42.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:42 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:42.658 DEBUG:teuthology.orchestra.run.smithi103:osd.0> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.0.service 2024-09-05T01:54:42.660 INFO:tasks.cephadm:Deploying osd.1 on smithi139 with /dev/vg_nvme/lv_4... 2024-09-05T01:54:42.661 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T01:54:42.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:42 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:42.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:42 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:42.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:42 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:42.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:42 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:54:42.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:42 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:42.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:42 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:42.899 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:54:43.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:43 smithi139 ceph-mon[34262]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:43.822 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:43 smithi103 ceph-mon[30902]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:43.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:43 smithi167 ceph-mon[35122]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:44.399 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:54:44 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0[41959]: 2024-09-05T01:54:44.246+0000 7fb4bdc77640 -1 osd.0 0 log_to_monitors true 2024-09-05T01:54:44.704 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:44 smithi139 ceph-mon[34262]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T01:54:44.823 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:44 smithi103 ceph-mon[30902]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T01:54:44.866 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:44 smithi167 ceph-mon[35122]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T01:54:45.290 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:54:45.315 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_4 2024-09-05T01:54:45.571 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:54:45.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:45 smithi139 ceph-mon[34262]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T01:54:45.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:45 smithi139 ceph-mon[34262]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T01:54:45.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:45 smithi139 ceph-mon[34262]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:54:45.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:45 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:45.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:45 smithi139 ceph-mon[34262]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:45.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:45 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:45.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:45 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:45.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:45 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:54:45.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:45 smithi103 ceph-mon[30902]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T01:54:45.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:45 smithi103 ceph-mon[30902]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T01:54:45.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:45 smithi103 ceph-mon[30902]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:54:45.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:45.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:45 smithi103 ceph-mon[30902]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:45.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:45.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:45.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:45 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:54:45.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:45 smithi167 ceph-mon[35122]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T01:54:45.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:45 smithi167 ceph-mon[35122]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T01:54:45.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:45 smithi167 ceph-mon[35122]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:54:45.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:45 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:45.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:45 smithi167 ceph-mon[35122]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:45.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:45 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:45.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:45 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:45.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:45 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:54:46.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.033+0000 7f2154707640 1 -- 172.21.15.139:0/241688501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c101cd0 msgr2=0x7f214c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:46.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.033+0000 7f2154707640 1 --2- 172.21.15.139:0/241688501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c101cd0 0x7f214c102150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f213c009900 tx=0x7f213c02a410 comp rx=0 tx=0).stop 2024-09-05T01:54:46.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.034+0000 7f2154707640 1 -- 172.21.15.139:0/241688501 shutdown_connections 2024-09-05T01:54:46.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.034+0000 7f2154707640 1 --2- 172.21.15.139:0/241688501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214c102690 0x7f214c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:46.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.034+0000 7f2154707640 1 --2- 172.21.15.139:0/241688501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c101cd0 0x7f214c102150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.034+0000 7f2154707640 1 --2- 172.21.15.139:0/241688501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f214c100ad0 0x7f214c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.034+0000 7f2154707640 1 -- 172.21.15.139:0/241688501 >> 172.21.15.139:0/241688501 conn(0x7f214c0fc280 msgr2=0x7f214c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.034+0000 7f2154707640 1 -- 172.21.15.139:0/241688501 shutdown_connections 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.034+0000 7f2154707640 1 -- 172.21.15.139:0/241688501 wait complete. 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.034+0000 7f2154707640 1 Processor -- start 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.034+0000 7f2154707640 1 -- start start 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f2154707640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c100ad0 0x7f214c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f2154707640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f214c101cd0 0x7f214c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f2154707640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214c102690 0x7f214c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f2154707640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f214c10c600 con 0x7f214c101cd0 2024-09-05T01:54:46.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f2154707640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f214c10c480 con 0x7f214c100ad0 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f2154707640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f214c10c780 con 0x7f214c102690 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21524a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c100ad0 0x7f214c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21524a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c100ad0 0x7f214c06aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41906/0 (socket says 172.21.15.139:41906) 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21524a3640 1 -- 172.21.15.139:0/2486204028 learned_addr learned my addr 172.21.15.139:0/2486204028 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f2151ca2640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f214c101cd0 0x7f214c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f2152ca4640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214c102690 0x7f214c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21524a3640 1 -- 172.21.15.139:0/2486204028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214c102690 msgr2=0x7f214c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21524a3640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214c102690 0x7f214c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21524a3640 1 -- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f214c101cd0 msgr2=0x7f214c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21524a3640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f214c101cd0 0x7f214c19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21524a3640 1 -- 172.21.15.139:0/2486204028 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c19fa40 con 0x7f214c100ad0 2024-09-05T01:54:46.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21524a3640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c100ad0 0x7f214c06aad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9c0 tx=0x7f214800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21437fe640 1 -- 172.21.15.139:0/2486204028 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214800cd80 con 0x7f214c100ad0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21437fe640 1 -- 172.21.15.139:0/2486204028 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21480035f0 con 0x7f214c100ad0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f21437fe640 1 -- 172.21.15.139:0/2486204028 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148010690 con 0x7f214c100ad0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.035+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c19fcd0 con 0x7f214c100ad0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.036+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c1a0190 con 0x7f214c100ad0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.037+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214c107f90 con 0x7f214c100ad0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.038+0000 7f21437fe640 1 -- 172.21.15.139:0/2486204028 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f21480037a0 con 0x7f214c100ad0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.038+0000 7f21437fe640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f21380a2af0 0x7f21380a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.039+0000 7f21437fe640 1 -- 172.21.15.139:0/2486204028 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f21480cd820 con 0x7f214c100ad0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.039+0000 7f2151ca2640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f21380a2af0 0x7f21380a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:54:46.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.040+0000 7f2151ca2640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f21380a2af0 0x7f21380a4fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f213c006010 tx=0x7f213c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:54:46.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.043+0000 7f21437fe640 1 -- 172.21.15.139:0/2486204028 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f214809c310 con 0x7f214c100ad0 2024-09-05T01:54:46.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:54:46.217+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f214c062630 con 0x7f21380a2af0 2024-09-05T01:54:46.538 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0[41959]: 2024-09-05T01:54:46.209+0000 7fb4ba40e640 -1 osd.0 0 waiting for initial osdmap 2024-09-05T01:54:46.538 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0[41959]: 2024-09-05T01:54:46.212+0000 7fb4b1a1a640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:54:46.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi103 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' 2024-09-05T01:54:46.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:54:46.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:54:46.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:54:46.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi103 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' 2024-09-05T01:54:46.868 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:54:46.869 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:54:46.869 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:54:46.869 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:46.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi103 2024-09-05T01:54:46.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-05T01:54:46.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:46.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:46.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:54:46.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:54:46.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T01:54:46.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:46.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:54:46.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969]' entity='osd.0' 2024-09-05T01:54:46.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:54:46.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:54:46.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:54:46.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:47.596 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:47 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:54:47.597 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:47 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:54:47.597 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:47 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:47.597 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:47 smithi139 ceph-mon[34262]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:47.597 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:47 smithi139 ceph-mon[34262]: osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] boot 2024-09-05T01:54:47.597 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:47 smithi139 ceph-mon[34262]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T01:54:47.597 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:47 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:47.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:47 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:54:47.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:47 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:54:47.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:47 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:47.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:47 smithi167 ceph-mon[35122]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:47.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:47 smithi167 ceph-mon[35122]: osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] boot 2024-09-05T01:54:47.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:47 smithi167 ceph-mon[35122]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T01:54:47.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:47 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:47.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:47 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:54:47.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:47 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:54:47.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:47 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:47.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:47 smithi103 ceph-mon[30902]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:54:47.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:47 smithi103 ceph-mon[30902]: osd.0 [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] boot 2024-09-05T01:54:47.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:47 smithi103 ceph-mon[30902]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T01:54:47.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:47 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:54:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:49 smithi139 ceph-mon[34262]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T01:54:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:49 smithi139 ceph-mon[34262]: from='client.? 172.21.15.139:0/192598438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0815a57-5a14-4d03-9045-36fd3649ab31"}]: dispatch 2024-09-05T01:54:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:49 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0815a57-5a14-4d03-9045-36fd3649ab31"}]: dispatch 2024-09-05T01:54:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:49 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0815a57-5a14-4d03-9045-36fd3649ab31"}]': finished 2024-09-05T01:54:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:49 smithi139 ceph-mon[34262]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T01:54:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:49 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:54:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:49 smithi139 ceph-mon[34262]: from='client.? 172.21.15.139:0/883555666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:54:49.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:49 smithi167 ceph-mon[35122]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T01:54:49.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:49 smithi167 ceph-mon[35122]: from='client.? 172.21.15.139:0/192598438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0815a57-5a14-4d03-9045-36fd3649ab31"}]: dispatch 2024-09-05T01:54:49.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:49 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0815a57-5a14-4d03-9045-36fd3649ab31"}]: dispatch 2024-09-05T01:54:49.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:49 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0815a57-5a14-4d03-9045-36fd3649ab31"}]': finished 2024-09-05T01:54:49.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:49 smithi167 ceph-mon[35122]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T01:54:49.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:49 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:54:49.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:49 smithi167 ceph-mon[35122]: from='client.? 172.21.15.139:0/883555666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:54:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:49 smithi103 ceph-mon[30902]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T01:54:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:49 smithi103 ceph-mon[30902]: from='client.? 172.21.15.139:0/192598438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0815a57-5a14-4d03-9045-36fd3649ab31"}]: dispatch 2024-09-05T01:54:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:49 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0815a57-5a14-4d03-9045-36fd3649ab31"}]: dispatch 2024-09-05T01:54:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:49 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0815a57-5a14-4d03-9045-36fd3649ab31"}]': finished 2024-09-05T01:54:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:49 smithi103 ceph-mon[30902]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T01:54:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:49 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:54:49.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:49 smithi103 ceph-mon[30902]: from='client.? 172.21.15.139:0/883555666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:54:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:50 smithi167 ceph-mon[35122]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:50.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:50 smithi139 ceph-mon[34262]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:50 smithi103 ceph-mon[30902]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:52 smithi167 ceph-mon[35122]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:52.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:52 smithi139 ceph-mon[34262]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:52.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:52 smithi103 ceph-mon[30902]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:54.263 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:54 smithi139 ceph-mon[34262]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:54 smithi167 ceph-mon[35122]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:54 smithi103 ceph-mon[30902]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:56 smithi167 ceph-mon[35122]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:56 smithi139 ceph-mon[34262]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:56 smithi103 ceph-mon[30902]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:57.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T01:54:57.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:57.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T01:54:57.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:57.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T01:54:57.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:54:58.533 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:58 smithi139 ceph-mon[34262]: Deploying daemon osd.1 on smithi139 2024-09-05T01:54:58.533 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:58 smithi139 ceph-mon[34262]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:58 smithi167 ceph-mon[35122]: Deploying daemon osd.1 on smithi139 2024-09-05T01:54:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:58 smithi167 ceph-mon[35122]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:58.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:58 smithi103 ceph-mon[30902]: Deploying daemon osd.1 on smithi139 2024-09-05T01:54:58.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:58 smithi103 ceph-mon[30902]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:54:59 smithi139 ceph-mon[34262]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:54:59 smithi167 ceph-mon[35122]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:54:59.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:54:59 smithi103 ceph-mon[30902]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:02.352 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:02 smithi103 ceph-mon[30902]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:02.352 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:02 smithi167 ceph-mon[35122]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:02.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:02 smithi139 ceph-mon[34262]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:02.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:03.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:03 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:03.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:03 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:03.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:03 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:03.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:03 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:03.610 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:03 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:03.610 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:03 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:04.335 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:04 smithi139 ceph-mon[34262]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:04.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:04 smithi167 ceph-mon[35122]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:04 smithi103 ceph-mon[30902]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:05.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:05.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:05.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:05.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:05.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:05.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:05.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:05.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:05.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:05.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:05.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:05.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:05.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:05.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:05.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:06.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:06 smithi139 ceph-mon[34262]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:06.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.405+0000 7f21437fe640 1 -- 172.21.15.139:0/2486204028 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f214c062630 con 0x7f21380a2af0 2024-09-05T01:55:06.406 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 1 on host 'smithi139' 2024-09-05T01:55:06.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f21380a2af0 msgr2=0x7f21380a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:06.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f21380a2af0 0x7f21380a4fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f213c006010 tx=0x7f213c000bc0 comp rx=0 tx=0).stop 2024-09-05T01:55:06.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c100ad0 msgr2=0x7f214c06aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:06.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c100ad0 0x7f214c06aad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9c0 tx=0x7f214800ee80 comp rx=0 tx=0).stop 2024-09-05T01:55:06.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 shutdown_connections 2024-09-05T01:55:06.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f21380a2af0 0x7f21380a4fb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:06.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214c102690 0x7f214c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:06.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f214c101cd0 0x7f214c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:06.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 --2- 172.21.15.139:0/2486204028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c100ad0 0x7f214c06aad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:06.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 >> 172.21.15.139:0/2486204028 conn(0x7f214c0fc280 msgr2=0x7f214c105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:55:06.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 shutdown_connections 2024-09-05T01:55:06.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:06.407+0000 7f2154707640 1 -- 172.21.15.139:0/2486204028 wait complete. 2024-09-05T01:55:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:06 smithi167 ceph-mon[35122]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:06 smithi103 ceph-mon[30902]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:07.077 DEBUG:teuthology.orchestra.run.smithi139:osd.1> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.1.service 2024-09-05T01:55:07.079 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2024-09-05T01:55:07.080 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T01:55:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:07 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:07 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:07 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:07.344 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:55:07.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:07 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:07.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:07 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:07.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:07 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:07.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:07 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:07.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:07 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:07.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:07 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:08.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:08 smithi139 ceph-mon[34262]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:08.449 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:08 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1[38685]: 2024-09-05T01:55:08.397+0000 7fe477a79640 -1 osd.1 0 log_to_monitors true 2024-09-05T01:55:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:08 smithi167 ceph-mon[35122]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:08 smithi103 ceph-mon[30902]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:09.416 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:09 smithi167 ceph-mon[35122]: from='osd.1 [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:55:09.416 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:09 smithi167 ceph-mon[35122]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:55:09.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:09 smithi139 ceph-mon[34262]: from='osd.1 [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:55:09.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:09 smithi139 ceph-mon[34262]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:55:09.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:09 smithi103 ceph-mon[30902]: from='osd.1 [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:55:09.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:09 smithi103 ceph-mon[30902]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:55:09.910 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:55:09.935 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-09-05T01:55:10.187 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:55:10.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:10.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T01:55:10.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='osd.1 [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:55:10.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T01:55:10.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:10.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:55:10.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:10.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:10.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:10.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.641+0000 7f1956dd8640 1 -- 172.21.15.167:0/588469012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1950102e00 msgr2=0x7f1950103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:10.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.641+0000 7f1956dd8640 1 --2- 172.21.15.167:0/588469012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1950102e00 0x7f1950103280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f193c009900 tx=0x7f193c02a410 comp rx=0 tx=0).stop 2024-09-05T01:55:10.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.641+0000 7f1956dd8640 1 -- 172.21.15.167:0/588469012 shutdown_connections 2024-09-05T01:55:10.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.641+0000 7f1956dd8640 1 --2- 172.21.15.167:0/588469012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19501037c0 0x7f195010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:10.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.641+0000 7f1956dd8640 1 --2- 172.21.15.167:0/588469012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1950102e00 0x7f1950103280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:10.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.641+0000 7f1956dd8640 1 --2- 172.21.15.167:0/588469012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950101c00 0x7f1950102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:10.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.641+0000 7f1956dd8640 1 -- 172.21.15.167:0/588469012 >> 172.21.15.167:0/588469012 conn(0x7f19500fd390 msgr2=0x7f19500ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:55:10.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.641+0000 7f1956dd8640 1 -- 172.21.15.167:0/588469012 shutdown_connections 2024-09-05T01:55:10.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.641+0000 7f1956dd8640 1 -- 172.21.15.167:0/588469012 wait complete. 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1956dd8640 1 Processor -- start 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1956dd8640 1 -- start start 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1956dd8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1950101c00 0x7f1950197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1956dd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950102e00 0x7f1950198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1956dd8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19501037c0 0x7f195019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1956dd8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f195010d730 con 0x7f1950102e00 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1956dd8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f195010d5b0 con 0x7f1950101c00 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1956dd8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f195010d8b0 con 0x7f19501037c0 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1955375640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19501037c0 0x7f195019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1955375640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19501037c0 0x7f195019f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44818/0 (socket says 172.21.15.167:44818) 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1954b74640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1950101c00 0x7f1950197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:10.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f1955375640 1 -- 172.21.15.167:0/1890689581 learned_addr learned my addr 172.21.15.167:0/1890689581 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.642+0000 7f194ffff640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950102e00 0x7f1950198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f1955375640 1 -- 172.21.15.167:0/1890689581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1950101c00 msgr2=0x7f1950197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f1955375640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1950101c00 0x7f1950197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f1955375640 1 -- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950102e00 msgr2=0x7f1950198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f1955375640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950102e00 0x7f1950198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f1955375640 1 -- 172.21.15.167:0/1890689581 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195019fa40 con 0x7f19501037c0 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f1955375640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19501037c0 0x7f195019f3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f194800eb10 tx=0x7f194800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f194dffb640 1 -- 172.21.15.167:0/1890689581 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948004b20 con 0x7f19501037c0 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f194dffb640 1 -- 172.21.15.167:0/1890689581 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1948004cc0 con 0x7f19501037c0 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195019fd30 con 0x7f19501037c0 2024-09-05T01:55:10.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f194dffb640 1 -- 172.21.15.167:0/1890689581 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948010690 con 0x7f19501037c0 2024-09-05T01:55:10.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.643+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19501a0230 con 0x7f19501037c0 2024-09-05T01:55:10.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.644+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19501090a0 con 0x7f19501037c0 2024-09-05T01:55:10.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:10.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='osd.1 [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:10.650 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='osd.1 [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:10.651 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:10.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.645+0000 7f194dffb640 1 -- 172.21.15.167:0/1890689581 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f1948005020 con 0x7f19501037c0 2024-09-05T01:55:10.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.646+0000 7f194dffb640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f19400a2ba0 0x7f19400a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:10.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.646+0000 7f194dffb640 1 -- 172.21.15.167:0/1890689581 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f19480cdb00 con 0x7f19501037c0 2024-09-05T01:55:10.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.646+0000 7f1954b74640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f19400a2ba0 0x7f19400a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:10.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.646+0000 7f1954b74640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f19400a2ba0 0x7f19400a5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1950102c60 tx=0x7f194400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:55:10.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.650+0000 7f194dffb640 1 -- 172.21.15.167:0/1890689581 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f194809c3d0 con 0x7f19501037c0 2024-09-05T01:55:10.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:10.821+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1950062630 con 0x7f19400a2ba0 2024-09-05T01:55:11.146 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1[38685]: 2024-09-05T01:55:10.847+0000 7fe4739fc640 -1 osd.1 0 waiting for initial osdmap 2024-09-05T01:55:11.146 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:10 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1[38685]: 2024-09-05T01:55:10.850+0000 7fe46b81c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:55:11.530 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi139 2024-09-05T01:55:11.530 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi139 to 17088M 2024-09-05T01:55:11.530 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:55:11.530 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T01:55:11.530 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:11.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:11.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:55:11.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:55:11.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:11.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:11 smithi167 ceph-mon[35122]: from='osd.1 ' entity='osd.1' 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi139 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi139 to 17088M 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:11 smithi139 ceph-mon[34262]: from='osd.1 ' entity='osd.1' 2024-09-05T01:55:11.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi139 2024-09-05T01:55:11.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi139 to 17088M 2024-09-05T01:55:11.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:55:11.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T01:55:11.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:11.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:11.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:55:11.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:55:11.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:11.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:11 smithi103 ceph-mon[30902]: from='osd.1 ' entity='osd.1' 2024-09-05T01:55:12.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:12 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:55:12.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:12 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:55:12.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:12 smithi167 ceph-mon[35122]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:55:12.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:12 smithi167 ceph-mon[35122]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:12.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:12 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:12.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:12 smithi167 ceph-mon[35122]: osd.1 [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] boot 2024-09-05T01:55:12.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:12 smithi167 ceph-mon[35122]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T01:55:12.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:12 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:12 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:55:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:12 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:55:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:12 smithi139 ceph-mon[34262]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:55:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:12 smithi139 ceph-mon[34262]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:12 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:12 smithi139 ceph-mon[34262]: osd.1 [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] boot 2024-09-05T01:55:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:12 smithi139 ceph-mon[34262]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T01:55:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:12 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:12 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:55:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:12 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:55:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:12 smithi103 ceph-mon[30902]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:55:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:12 smithi103 ceph-mon[30902]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:55:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:12 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:12 smithi103 ceph-mon[30902]: osd.1 [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] boot 2024-09-05T01:55:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:12 smithi103 ceph-mon[30902]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T01:55:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:12 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:55:13.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:13 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/3238623404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13eeb8f3-8b22-415b-abd9-d8d054bc1093"}]: dispatch 2024-09-05T01:55:13.448 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:13 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13eeb8f3-8b22-415b-abd9-d8d054bc1093"}]: dispatch 2024-09-05T01:55:13.448 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:13 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13eeb8f3-8b22-415b-abd9-d8d054bc1093"}]': finished 2024-09-05T01:55:13.448 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:13 smithi167 ceph-mon[35122]: osdmap e14: 3 total, 2 up, 3 in 2024-09-05T01:55:13.448 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:13.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:13 smithi139 ceph-mon[34262]: from='client.? 172.21.15.167:0/3238623404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13eeb8f3-8b22-415b-abd9-d8d054bc1093"}]: dispatch 2024-09-05T01:55:13.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:13 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13eeb8f3-8b22-415b-abd9-d8d054bc1093"}]: dispatch 2024-09-05T01:55:13.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:13 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13eeb8f3-8b22-415b-abd9-d8d054bc1093"}]': finished 2024-09-05T01:55:13.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:13 smithi139 ceph-mon[34262]: osdmap e14: 3 total, 2 up, 3 in 2024-09-05T01:55:13.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:13 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/3238623404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13eeb8f3-8b22-415b-abd9-d8d054bc1093"}]: dispatch 2024-09-05T01:55:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:13 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13eeb8f3-8b22-415b-abd9-d8d054bc1093"}]: dispatch 2024-09-05T01:55:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:13 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13eeb8f3-8b22-415b-abd9-d8d054bc1093"}]': finished 2024-09-05T01:55:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:13 smithi103 ceph-mon[30902]: osdmap e14: 3 total, 2 up, 3 in 2024-09-05T01:55:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:14 smithi167 ceph-mon[35122]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:14 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/1773190811' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:55:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:14 smithi139 ceph-mon[34262]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:14 smithi139 ceph-mon[34262]: from='client.? 172.21.15.167:0/1773190811' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:55:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:14 smithi103 ceph-mon[30902]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:14 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/1773190811' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:55:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:16 smithi167 ceph-mon[35122]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:16 smithi139 ceph-mon[34262]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:16 smithi103 ceph-mon[30902]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:18.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:18 smithi167 ceph-mon[35122]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:18 smithi139 ceph-mon[34262]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:18 smithi103 ceph-mon[30902]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:20.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:20 smithi167 ceph-mon[35122]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:20 smithi139 ceph-mon[34262]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:20 smithi103 ceph-mon[30902]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:22 smithi167 ceph-mon[35122]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:22 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T01:55:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:22 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:22 smithi139 ceph-mon[34262]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:22.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:22 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T01:55:22.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:22 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:22.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:22 smithi103 ceph-mon[30902]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:22.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:22 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T01:55:22.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:22 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:23.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:23 smithi167 ceph-mon[35122]: Deploying daemon osd.2 on smithi167 2024-09-05T01:55:23.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:23 smithi139 ceph-mon[34262]: Deploying daemon osd.2 on smithi167 2024-09-05T01:55:23.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:23 smithi103 ceph-mon[30902]: Deploying daemon osd.2 on smithi167 2024-09-05T01:55:24.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:24 smithi167 ceph-mon[35122]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:24.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:24 smithi139 ceph-mon[34262]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:24 smithi103 ceph-mon[30902]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:26 smithi167 ceph-mon[35122]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:26.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:26 smithi139 ceph-mon[34262]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:26.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:26 smithi103 ceph-mon[30902]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:28.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:28 smithi167 ceph-mon[35122]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:28 smithi139 ceph-mon[34262]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:28 smithi103 ceph-mon[30902]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:29.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:29.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:29.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:29.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:29.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:29.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:29 smithi167 ceph-mon[35122]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:29.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:29.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:29.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:29.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:29.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:29.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:29 smithi139 ceph-mon[34262]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:29.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:29.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:29.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:29.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:29.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:29.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:29 smithi103 ceph-mon[30902]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:30.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.611+0000 7f194dffb640 1 -- 172.21.15.167:0/1890689581 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1950062630 con 0x7f19400a2ba0 2024-09-05T01:55:30.613 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-09-05T01:55:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f19400a2ba0 msgr2=0x7f19400a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f19400a2ba0 0x7f19400a5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1950102c60 tx=0x7f194400b390 comp rx=0 tx=0).stop 2024-09-05T01:55:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19501037c0 msgr2=0x7f195019f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19501037c0 0x7f195019f3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f194800eb10 tx=0x7f194800c2d0 comp rx=0 tx=0).stop 2024-09-05T01:55:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 shutdown_connections 2024-09-05T01:55:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f19400a2ba0 0x7f19400a5060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19501037c0 0x7f195019f3d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950102e00 0x7f1950198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 --2- 172.21.15.167:0/1890689581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1950101c00 0x7f1950197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 >> 172.21.15.167:0/1890689581 conn(0x7f19500fd390 msgr2=0x7f1950106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:55:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 shutdown_connections 2024-09-05T01:55:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:55:30.614+0000 7f1956dd8640 1 -- 172.21.15.167:0/1890689581 wait complete. 2024-09-05T01:55:30.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:30 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:30.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:30 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:30.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:30 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:31.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:30 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:31.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:30 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:31.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:30 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:31.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:31.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:31.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:31.230 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.2.service 2024-09-05T01:55:31.232 INFO:tasks.cephadm:Deploying osd.3 on smithi103 with /dev/vg_nvme/lv_3... 2024-09-05T01:55:31.232 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T01:55:31.491 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:55:31.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:31 smithi167 ceph-mon[35122]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:31.864 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:31 smithi103 ceph-mon[30902]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:32.135 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:31 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2[40055]: 2024-09-05T01:55:31.772+0000 7fc3e4ae4640 -1 osd.2 0 log_to_monitors true 2024-09-05T01:55:32.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:31 smithi139 ceph-mon[34262]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:32.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:32 smithi167 ceph-mon[35122]: from='osd.2 [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:55:32.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:32 smithi167 ceph-mon[35122]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:55:32.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:32 smithi103 ceph-mon[30902]: from='osd.2 [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:55:32.883 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:32 smithi103 ceph-mon[30902]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:55:33.047 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:32 smithi139 ceph-mon[34262]: from='osd.2 [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:55:33.047 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:32 smithi139 ceph-mon[34262]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:55:33.982 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:33 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2[40055]: 2024-09-05T01:55:33.673+0000 7fc3e0a67640 -1 osd.2 0 waiting for initial osdmap 2024-09-05T01:55:33.982 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:33 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2[40055]: 2024-09-05T01:55:33.676+0000 7fc3dc88f640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:55:33.983 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:33 smithi167 ceph-mon[35122]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T01:55:33.983 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:33 smithi167 ceph-mon[35122]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T01:55:33.983 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:33.983 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:33 smithi167 ceph-mon[35122]: from='osd.2 [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:55:33.983 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:33 smithi167 ceph-mon[35122]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:55:33.983 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:33 smithi167 ceph-mon[35122]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:34.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:33 smithi103 ceph-mon[30902]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T01:55:34.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:33 smithi103 ceph-mon[30902]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T01:55:34.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:34.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:33 smithi103 ceph-mon[30902]: from='osd.2 [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:55:34.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:33 smithi103 ceph-mon[30902]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:55:34.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:33 smithi103 ceph-mon[30902]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:34.038 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:55:34.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-09-05T01:55:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:33 smithi139 ceph-mon[34262]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T01:55:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:33 smithi139 ceph-mon[34262]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T01:55:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:33 smithi139 ceph-mon[34262]: from='osd.2 [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:55:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:33 smithi139 ceph-mon[34262]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:55:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:33 smithi139 ceph-mon[34262]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:55:34.330 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:55:34.748 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:55:34.748 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T01:55:34.748 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:34.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:34.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:34.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:34.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:34.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:34.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:34.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:34.749 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:34 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:34.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.830+0000 7ff5d22e6640 1 -- 172.21.15.103:0/1579130464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc1037c0 msgr2=0x7ff5cc10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:34.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.830+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/1579130464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc1037c0 0x7ff5cc10a050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009ca0 tx=0x7ff5c402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:55:34.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.830+0000 7ff5d22e6640 1 -- 172.21.15.103:0/1579130464 shutdown_connections 2024-09-05T01:55:34.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.830+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/1579130464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc1037c0 0x7ff5cc10a050 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.830+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/1579130464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5cc102e00 0x7ff5cc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.830+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/1579130464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5cc101c00 0x7ff5cc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.830+0000 7ff5d22e6640 1 -- 172.21.15.103:0/1579130464 >> 172.21.15.103:0/1579130464 conn(0x7ff5cc0fd390 msgr2=0x7ff5cc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:55:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.830+0000 7ff5d22e6640 1 -- 172.21.15.103:0/1579130464 shutdown_connections 2024-09-05T01:55:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.830+0000 7ff5d22e6640 1 -- 172.21.15.103:0/1579130464 wait complete. 2024-09-05T01:55:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5d22e6640 1 Processor -- start 2024-09-05T01:55:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5d22e6640 1 -- start start 2024-09-05T01:55:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5d22e6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5cc101c00 0x7ff5cc197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5d22e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc102e00 0x7ff5cc198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5d22e6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5cc1037c0 0x7ff5cc19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5d22e6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc10d710 con 0x7ff5cc102e00 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5d22e6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc10d590 con 0x7ff5cc101c00 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5d22e6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc10d890 con 0x7ff5cc1037c0 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc102e00 0x7ff5cc198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc102e00 0x7ff5cc198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43776/0 (socket says 172.21.15.103:43776) 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5cbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5cc101c00 0x7ff5cc197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5cb7fe640 1 -- 172.21.15.103:0/3756969100 learned_addr learned my addr 172.21.15.103:0/3756969100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.831+0000 7ff5d0883640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5cc1037c0 0x7ff5cc19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5cb7fe640 1 -- 172.21.15.103:0/3756969100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5cc1037c0 msgr2=0x7ff5cc19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5cb7fe640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5cc1037c0 0x7ff5cc19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5cb7fe640 1 -- 172.21.15.103:0/3756969100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5cc101c00 msgr2=0x7ff5cc197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5cb7fe640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5cc101c00 0x7ff5cc197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5cb7fe640 1 -- 172.21.15.103:0/3756969100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc19f9f0 con 0x7ff5cc102e00 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5cbfff640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5cc101c00 0x7ff5cc197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5cb7fe640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc102e00 0x7ff5cc198300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000d950 tx=0x7ff5c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5c97fa640 1 -- 172.21.15.103:0/3756969100 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0017070 con 0x7ff5cc102e00 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5c97fa640 1 -- 172.21.15.103:0/3756969100 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5c0009c30 con 0x7ff5cc102e00 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5cc19fce0 con 0x7ff5cc102e00 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5c97fa640 1 -- 172.21.15.103:0/3756969100 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000f620 con 0x7ff5cc102e00 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.832+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5cc1a0210 con 0x7ff5cc102e00 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.833+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5cc109050 con 0x7ff5cc102e00 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.834+0000 7ff5c97fa640 1 -- 172.21.15.103:0/3756969100 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7ff5c00109a0 con 0x7ff5cc102e00 2024-09-05T01:55:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.835+0000 7ff5c97fa640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b40a2ba0 0x7ff5b40a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:34.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.835+0000 7ff5cbfff640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b40a2ba0 0x7ff5b40a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:34.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.835+0000 7ff5c97fa640 1 -- 172.21.15.103:0/3756969100 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2851+0+0 (secure 0 0 0) 0x7ff5c00cd480 con 0x7ff5cc102e00 2024-09-05T01:55:34.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.835+0000 7ff5cbfff640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b40a2ba0 0x7ff5b40a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc102c60 tx=0x7ff5bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:55:34.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:34.839+0000 7ff5c97fa640 1 -- 172.21.15.103:0/3756969100 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5c009b3c0 con 0x7ff5cc102e00 2024-09-05T01:55:35.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:35.015+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff5cc062630 con 0x7ff5b40a2ba0 2024-09-05T01:55:35.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:35.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:34 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:35.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:34 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:36.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:55:36.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:55:36.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi167 2024-09-05T01:55:36.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi167 to 17088M 2024-09-05T01:55:36.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: osd.2 [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] boot 2024-09-05T01:55:36.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: osdmap e17: 3 total, 3 up, 3 in 2024-09-05T01:55:36.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:36.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:55:36.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:55:36.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:55:36.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:36.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: pgmap v72: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:36.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:35 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi167 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi167 to 17088M 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: osd.2 [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] boot 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: osdmap e17: 3 total, 3 up, 3 in 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: pgmap v72: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:36.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:35 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:55:36.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:55:36.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:55:36.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi167 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi167 to 17088M 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: osd.2 [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] boot 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: osdmap e17: 3 total, 3 up, 3 in 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: pgmap v72: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:36.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:35 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:55:36.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T01:55:36.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:36 smithi103 ceph-mon[30902]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T01:55:36.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:55:37.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:36 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T01:55:37.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:36 smithi167 ceph-mon[35122]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T01:55:37.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:36 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:55:37.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:36 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T01:55:37.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:36 smithi139 ceph-mon[34262]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T01:55:37.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:36 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:55:37.148 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:55:37 smithi103 sudo[46409]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:55:37.149 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:55:37 smithi103 sudo[46409]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:37.149 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:55:37 smithi103 sudo[46409]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:37.149 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:55:37 smithi103 sudo[46409]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:37.648 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:55:37 smithi103 sudo[46435]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:55:37.648 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:55:37 smithi103 sudo[46435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:37.648 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:55:37 smithi103 sudo[46435]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:37.649 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 01:55:37 smithi103 sudo[46435]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:37.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:37.701+0000 7ff5c97fa640 1 -- 172.21.15.103:0/3756969100 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff5c0097a30 con 0x7ff5cc102e00 2024-09-05T01:55:37.835 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:37 smithi139 sudo[42816]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:55:37.835 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:37 smithi139 sudo[42816]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:37.835 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:37 smithi139 sudo[42816]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:37.836 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:37 smithi139 sudo[42816]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:37.836 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T01:55:37.836 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:37 smithi139 ceph-mon[34262]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T01:55:37.836 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:37 smithi139 ceph-mon[34262]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:37.836 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:37.836 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:37 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2032000902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80e9c6a8-bf1d-45a1-9e94-6abba2659c17"}]: dispatch 2024-09-05T01:55:37.836 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:37 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2032000902' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80e9c6a8-bf1d-45a1-9e94-6abba2659c17"}]': finished 2024-09-05T01:55:37.836 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:37 smithi139 ceph-mon[34262]: osdmap e20: 4 total, 3 up, 4 in 2024-09-05T01:55:37.836 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:55:38.094 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T01:55:38.094 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:37 smithi103 ceph-mon[30902]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T01:55:38.094 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:37 smithi103 ceph-mon[30902]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:38.094 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:38.094 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:37 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2032000902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80e9c6a8-bf1d-45a1-9e94-6abba2659c17"}]: dispatch 2024-09-05T01:55:38.094 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:37 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2032000902' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80e9c6a8-bf1d-45a1-9e94-6abba2659c17"}]': finished 2024-09-05T01:55:38.095 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:37 smithi103 ceph-mon[30902]: osdmap e20: 4 total, 3 up, 4 in 2024-09-05T01:55:38.095 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:55:38.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T01:55:38.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:37 smithi167 ceph-mon[35122]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T01:55:38.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:37 smithi167 ceph-mon[35122]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:38.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:38.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:37 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2032000902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80e9c6a8-bf1d-45a1-9e94-6abba2659c17"}]: dispatch 2024-09-05T01:55:38.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:37 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2032000902' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80e9c6a8-bf1d-45a1-9e94-6abba2659c17"}]': finished 2024-09-05T01:55:38.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:37 smithi167 ceph-mon[35122]: osdmap e20: 4 total, 3 up, 4 in 2024-09-05T01:55:38.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:55:38.146 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:37 smithi139 sudo[42819]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:55:38.146 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:37 smithi139 sudo[42819]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:38.146 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:37 smithi139 sudo[42819]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:38.146 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 01:55:37 smithi139 sudo[42819]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:38.361 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:38 smithi167 sudo[44133]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:55:38.361 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:38 smithi167 sudo[44133]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:38.361 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:38 smithi167 sudo[44133]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:38.361 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:38 smithi167 sudo[44133]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:38.635 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:38 smithi167 sudo[44136]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:55:38.635 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:38 smithi167 sudo[44136]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:38.635 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:38 smithi167 sudo[44136]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:38.635 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 01:55:38 smithi167 sudo[44136]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:38.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 sudo[46534]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:55:38.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 sudo[46534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:38.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 sudo[46534]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:38.892 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 sudo[46534]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:39.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 sudo[46538]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:55:39.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 sudo[46538]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:39.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 sudo[46538]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:39.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 sudo[46538]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:39.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 ceph-mon[30902]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T01:55:39.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:39.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2866926788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:55:39.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:39.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:38 smithi103 ceph-mon[30902]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:55:39.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:38 smithi167 ceph-mon[35122]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T01:55:39.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:38 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:39.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:38 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2866926788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:55:39.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:38 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:39.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:38 smithi167 ceph-mon[35122]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:55:39.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 sudo[44139]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:55:39.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 sudo[44139]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:39.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 sudo[44139]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:39.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 sudo[44139]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:39.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:38 smithi139 ceph-mon[34262]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T01:55:39.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:38 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:39.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:38 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2866926788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:55:39.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:38 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:39.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:38 smithi139 ceph-mon[34262]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:55:39.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 sudo[44142]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:55:39.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 sudo[44142]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:39.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 sudo[44142]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:39.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 sudo[44142]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:39.894 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 sudo[42822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:55:39.894 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 sudo[42822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:39.894 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 sudo[42822]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:39.894 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 sudo[42822]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 sudo[42825]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 sudo[42825]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 sudo[42825]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 sudo[42825]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:55:40.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:55:40.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:55:40.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:40.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:55:40.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:55:40.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:55:40.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:39 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:55:40.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:55:40.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:55:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:39 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:55:40.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:55:40.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:55:40.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:55:40.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:55:40.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:55:40.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:55:40.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:55:40.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:55:40.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:40.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:55:40.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:55:40.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:55:40.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:39 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:55:41.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:40 smithi167 ceph-mon[35122]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:55:41.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:40 smithi167 ceph-mon[35122]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:55:41.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:40 smithi139 ceph-mon[34262]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:55:41.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:40 smithi139 ceph-mon[34262]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:55:41.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:40 smithi103 ceph-mon[30902]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:55:41.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:40 smithi103 ceph-mon[30902]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:55:42.329 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:41 smithi103 ceph-mon[30902]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:42.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:41 smithi167 ceph-mon[35122]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:42.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:41 smithi139 ceph-mon[34262]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:44.345 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:44 smithi139 ceph-mon[34262]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 6.5 KiB/s rd, 331 KiB/s wr, 22 op/s 2024-09-05T01:55:44.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:44 smithi167 ceph-mon[35122]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 6.5 KiB/s rd, 331 KiB/s wr, 22 op/s 2024-09-05T01:55:44.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:44 smithi103 ceph-mon[30902]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 6.5 KiB/s rd, 331 KiB/s wr, 22 op/s 2024-09-05T01:55:46.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:46 smithi167 ceph-mon[35122]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 243 KiB/s wr, 16 op/s 2024-09-05T01:55:46.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T01:55:46.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:46.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:46 smithi139 ceph-mon[34262]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 243 KiB/s wr, 16 op/s 2024-09-05T01:55:46.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T01:55:46.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:46.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:46 smithi103 ceph-mon[30902]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 243 KiB/s wr, 16 op/s 2024-09-05T01:55:46.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T01:55:46.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:47.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:47 smithi167 ceph-mon[35122]: Deploying daemon osd.3 on smithi103 2024-09-05T01:55:47.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:47 smithi139 ceph-mon[34262]: Deploying daemon osd.3 on smithi103 2024-09-05T01:55:47.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:47 smithi103 ceph-mon[30902]: Deploying daemon osd.3 on smithi103 2024-09-05T01:55:48.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:48 smithi103 ceph-mon[30902]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-09-05T01:55:48.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:48 smithi167 ceph-mon[35122]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-09-05T01:55:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:48 smithi139 ceph-mon[34262]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-09-05T01:55:50.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:50 smithi167 ceph-mon[35122]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 159 KiB/s wr, 10 op/s 2024-09-05T01:55:50.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:50 smithi139 ceph-mon[34262]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 159 KiB/s wr, 10 op/s 2024-09-05T01:55:50.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:50 smithi103 ceph-mon[30902]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 159 KiB/s wr, 10 op/s 2024-09-05T01:55:52.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:52 smithi103 ceph-mon[30902]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:55:52.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:52.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:52.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:52.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:52 smithi167 ceph-mon[35122]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:55:52.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:52.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:52.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:52.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:52 smithi139 ceph-mon[34262]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:55:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:54.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:54 smithi167 ceph-mon[35122]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:55:54.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:54 smithi139 ceph-mon[34262]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:55:54.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:54 smithi103 ceph-mon[30902]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:55:55.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.175+0000 7ff5c97fa640 1 -- 172.21.15.103:0/3756969100 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff5cc062630 con 0x7ff5b40a2ba0 2024-09-05T01:55:55.177 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 3 on host 'smithi103' 2024-09-05T01:55:55.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b40a2ba0 msgr2=0x7ff5b40a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b40a2ba0 0x7ff5b40a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc102c60 tx=0x7ff5bc00b390 comp rx=0 tx=0).stop 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc102e00 msgr2=0x7ff5cc198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc102e00 0x7ff5cc198300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000d950 tx=0x7ff5c000de10 comp rx=0 tx=0).stop 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 shutdown_connections 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b40a2ba0 0x7ff5b40a5060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5cc1037c0 0x7ff5cc19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5cc102e00 0x7ff5cc198300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 --2- 172.21.15.103:0/3756969100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5cc101c00 0x7ff5cc197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 >> 172.21.15.103:0/3756969100 conn(0x7ff5cc0fd390 msgr2=0x7ff5cc106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 shutdown_connections 2024-09-05T01:55:55.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:55:55.177+0000 7ff5d22e6640 1 -- 172.21.15.103:0/3756969100 wait complete. 2024-09-05T01:55:55.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:55 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:55 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:55 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:55 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:55 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:55.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:55 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:55.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:55 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:55.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:55 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:55.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:55 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:55.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:55 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:55 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:55 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:55 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:55 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:55.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:55 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:55.802 DEBUG:teuthology.orchestra.run.smithi103:osd.3> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.3.service 2024-09-05T01:55:55.804 INFO:tasks.cephadm:Deploying osd.4 on smithi139 with /dev/vg_nvme/lv_3... 2024-09-05T01:55:55.805 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T01:55:56.058 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:55:56.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:56 smithi103 ceph-mon[30902]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:56.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:56 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:56.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:56 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:56.374 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:56 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:56.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:56 smithi139 ceph-mon[34262]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:56.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:56 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:56.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:56 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:56.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:56 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:56 smithi167 ceph-mon[35122]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:55:56.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:56.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:57.899 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 01:55:57 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3[48470]: 2024-09-05T01:55:57.518+0000 7f2aebb7e640 -1 osd.3 0 log_to_monitors true 2024-09-05T01:55:58.298 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:58 smithi103 ceph-mon[30902]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:58.298 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:58 smithi103 ceph-mon[30902]: from='osd.3 [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T01:55:58.298 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:58 smithi103 ceph-mon[30902]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T01:55:58.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:58 smithi139 ceph-mon[34262]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:58.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:58 smithi139 ceph-mon[34262]: from='osd.3 [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T01:55:58.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:58 smithi139 ceph-mon[34262]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T01:55:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:58 smithi167 ceph-mon[35122]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:55:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:58 smithi167 ceph-mon[35122]: from='osd.3 [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T01:55:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:58 smithi167 ceph-mon[35122]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T01:55:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:55:58.730 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_3 2024-09-05T01:55:58.987 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: osdmap e21: 4 total, 3 up, 4 in 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='osd.3 [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:59.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:59.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:55:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.523+0000 7f6065f8f640 1 -- 172.21.15.139:0/57443077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060105050 msgr2=0x7f6060107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:59.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.523+0000 7f6065f8f640 1 --2- 172.21.15.139:0/57443077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060105050 0x7f6060107440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6054009900 tx=0x7f605402a410 comp rx=0 tx=0).stop 2024-09-05T01:55:59.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.523+0000 7f6065f8f640 1 -- 172.21.15.139:0/57443077 shutdown_connections 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.523+0000 7f6065f8f640 1 --2- 172.21.15.139:0/57443077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6060107980 0x7f6060109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.523+0000 7f6065f8f640 1 --2- 172.21.15.139:0/57443077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060105050 0x7f6060107440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.523+0000 7f6065f8f640 1 --2- 172.21.15.139:0/57443077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6060102720 0x7f6060104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.523+0000 7f6065f8f640 1 -- 172.21.15.139:0/57443077 >> 172.21.15.139:0/57443077 conn(0x7f60600fc2a0 msgr2=0x7f60600fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.523+0000 7f6065f8f640 1 -- 172.21.15.139:0/57443077 shutdown_connections 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.523+0000 7f6065f8f640 1 -- 172.21.15.139:0/57443077 wait complete. 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f6065f8f640 1 Processor -- start 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f6065f8f640 1 -- start start 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f6065f8f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060102720 0x7f6060197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f6065f8f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6060105050 0x7f60601983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f6065f8f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6060107980 0x7f606019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f6065f8f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f606010d740 con 0x7f6060107980 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f6065f8f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f606010d5c0 con 0x7f6060102720 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f6065f8f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f606010d8c0 con 0x7f6060105050 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f605f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060102720 0x7f6060197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f605ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6060107980 0x7f606019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f605f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060102720 0x7f6060197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58582/0 (socket says 172.21.15.139:58582) 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f605f7fe640 1 -- 172.21.15.139:0/2392451724 learned_addr learned my addr 172.21.15.139:0/2392451724 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.524+0000 7f605effd640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6060105050 0x7f60601983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605f7fe640 1 -- 172.21.15.139:0/2392451724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6060105050 msgr2=0x7f60601983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605f7fe640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6060105050 0x7f60601983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605f7fe640 1 -- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6060107980 msgr2=0x7f606019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:55:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605f7fe640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6060107980 0x7f606019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:55:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605f7fe640 1 -- 172.21.15.139:0/2392451724 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606019fac0 con 0x7f6060102720 2024-09-05T01:55:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605ffff640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6060107980 0x7f606019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:55:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605f7fe640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060102720 0x7f6060197e90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f605000e9f0 tx=0x7f605000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:55:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605cff9640 1 -- 172.21.15.139:0/2392451724 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605000ccc0 con 0x7f6060102720 2024-09-05T01:55:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605cff9640 1 -- 172.21.15.139:0/2392451724 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f605000ce60 con 0x7f6060102720 2024-09-05T01:55:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606019fdb0 con 0x7f6060102720 2024-09-05T01:55:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f605cff9640 1 -- 172.21.15.139:0/2392451724 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050010430 con 0x7f6060102720 2024-09-05T01:55:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.525+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60601a01b0 con 0x7f6060102720 2024-09-05T01:55:59.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.526+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60600fdd90 con 0x7f6060102720 2024-09-05T01:55:59.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.527+0000 7f605cff9640 1 -- 172.21.15.139:0/2392451724 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6050005020 con 0x7f6060102720 2024-09-05T01:55:59.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.528+0000 7f605cff9640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60480a2bb0 0x7f60480a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:55:59.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.528+0000 7f605cff9640 1 -- 172.21.15.139:0/2392451724 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3389+0+0 (secure 0 0 0) 0x7f60500ce120 con 0x7f6060102720 2024-09-05T01:55:59.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.528+0000 7f605effd640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60480a2bb0 0x7f60480a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:55:59.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.528+0000 7f605effd640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60480a2bb0 0x7f60480a5070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6054000c00 tx=0x7f60540314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:55:59.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.532+0000 7f605cff9640 1 -- 172.21.15.139:0/2392451724 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f605009c3c0 con 0x7f6060102720 2024-09-05T01:55:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T01:55:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: osdmap e21: 4 total, 3 up, 4 in 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='osd.3 [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:55:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T01:55:59.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: osdmap e21: 4 total, 3 up, 4 in 2024-09-05T01:55:59.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:55:59.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='osd.3 [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:55:59.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:55:59.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:59.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:55:59.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:55:59.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:55:59.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:55:59.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:55:59.709+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6060062630 con 0x7f60480a2bb0 2024-09-05T01:56:00.148 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3[48470]: 2024-09-05T01:55:59.822+0000 7f2ae8315640 -1 osd.3 0 waiting for initial osdmap 2024-09-05T01:56:00.148 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 01:55:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3[48470]: 2024-09-05T01:55:59.828+0000 7f2adf921640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:56:00.372 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi103 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:00.373 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:00 smithi139 ceph-mon[34262]: from='osd.3 ' entity='osd.3' 2024-09-05T01:56:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi103 2024-09-05T01:56:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-05T01:56:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:56:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:56:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T01:56:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:56:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:56:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T01:56:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:00 smithi167 ceph-mon[35122]: from='osd.3 ' entity='osd.3' 2024-09-05T01:56:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi103 2024-09-05T01:56:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-05T01:56:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:00 smithi103 ceph-mon[30902]: from='osd.3 ' entity='osd.3' 2024-09-05T01:56:01.390 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:01 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:56:01.390 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:01 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:56:01.390 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:01 smithi139 ceph-mon[34262]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:56:01.390 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:01 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:01.391 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:01 smithi139 ceph-mon[34262]: osd.3 [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] boot 2024-09-05T01:56:01.391 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:01 smithi139 ceph-mon[34262]: osdmap e24: 4 total, 4 up, 4 in 2024-09-05T01:56:01.391 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:01 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:01.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:01 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:56:01.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:01 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:56:01.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:01 smithi167 ceph-mon[35122]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:56:01.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:01.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:01 smithi167 ceph-mon[35122]: osd.3 [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] boot 2024-09-05T01:56:01.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:01 smithi167 ceph-mon[35122]: osdmap e24: 4 total, 4 up, 4 in 2024-09-05T01:56:01.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:01.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:01 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:56:01.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:01 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:56:01.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:01 smithi103 ceph-mon[30902]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:56:01.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:01.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:01 smithi103 ceph-mon[30902]: osd.3 [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] boot 2024-09-05T01:56:01.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:01 smithi103 ceph-mon[30902]: osdmap e24: 4 total, 4 up, 4 in 2024-09-05T01:56:01.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:56:02.550 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:02 smithi103 ceph-mon[30902]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:56:02.550 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:02 smithi103 ceph-mon[30902]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T01:56:02.550 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:02 smithi103 ceph-mon[30902]: from='client.? 172.21.15.139:0/182542724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed03c495-8ced-4b29-af59-3fec3dab74fb"}]: dispatch 2024-09-05T01:56:02.550 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:02 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed03c495-8ced-4b29-af59-3fec3dab74fb"}]: dispatch 2024-09-05T01:56:02.550 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:02 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed03c495-8ced-4b29-af59-3fec3dab74fb"}]': finished 2024-09-05T01:56:02.550 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:02 smithi103 ceph-mon[30902]: osdmap e26: 5 total, 4 up, 5 in 2024-09-05T01:56:02.550 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:02.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:02 smithi139 ceph-mon[34262]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:56:02.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:02 smithi139 ceph-mon[34262]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T01:56:02.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:02 smithi139 ceph-mon[34262]: from='client.? 172.21.15.139:0/182542724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed03c495-8ced-4b29-af59-3fec3dab74fb"}]: dispatch 2024-09-05T01:56:02.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:02 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed03c495-8ced-4b29-af59-3fec3dab74fb"}]: dispatch 2024-09-05T01:56:02.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:02 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed03c495-8ced-4b29-af59-3fec3dab74fb"}]': finished 2024-09-05T01:56:02.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:02 smithi139 ceph-mon[34262]: osdmap e26: 5 total, 4 up, 5 in 2024-09-05T01:56:02.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:02 smithi167 ceph-mon[35122]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:56:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:02 smithi167 ceph-mon[35122]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T01:56:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:02 smithi167 ceph-mon[35122]: from='client.? 172.21.15.139:0/182542724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed03c495-8ced-4b29-af59-3fec3dab74fb"}]: dispatch 2024-09-05T01:56:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:02 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed03c495-8ced-4b29-af59-3fec3dab74fb"}]: dispatch 2024-09-05T01:56:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:02 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed03c495-8ced-4b29-af59-3fec3dab74fb"}]': finished 2024-09-05T01:56:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:02 smithi167 ceph-mon[35122]: osdmap e26: 5 total, 4 up, 5 in 2024-09-05T01:56:02.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:03.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:03 smithi167 ceph-mon[35122]: from='client.? 172.21.15.139:0/3516004464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:56:03.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:03 smithi167 ceph-mon[35122]: osdmap e27: 5 total, 4 up, 5 in 2024-09-05T01:56:03.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:03 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:03.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:03 smithi139 ceph-mon[34262]: from='client.? 172.21.15.139:0/3516004464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:56:03.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:03 smithi139 ceph-mon[34262]: osdmap e27: 5 total, 4 up, 5 in 2024-09-05T01:56:03.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:03 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:03.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:03 smithi103 ceph-mon[30902]: from='client.? 172.21.15.139:0/3516004464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:56:03.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:03 smithi103 ceph-mon[30902]: osdmap e27: 5 total, 4 up, 5 in 2024-09-05T01:56:03.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:03 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:04 smithi167 ceph-mon[35122]: pgmap v95: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-09-05T01:56:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:04 smithi167 ceph-mon[35122]: osdmap e28: 5 total, 4 up, 5 in 2024-09-05T01:56:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:04.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:04 smithi139 ceph-mon[34262]: pgmap v95: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-09-05T01:56:04.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:04 smithi139 ceph-mon[34262]: osdmap e28: 5 total, 4 up, 5 in 2024-09-05T01:56:04.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:04.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:04 smithi103 ceph-mon[30902]: pgmap v95: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-09-05T01:56:04.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:04 smithi103 ceph-mon[30902]: osdmap e28: 5 total, 4 up, 5 in 2024-09-05T01:56:04.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:06.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:06 smithi139 ceph-mon[34262]: pgmap v98: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T01:56:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:06 smithi167 ceph-mon[35122]: pgmap v98: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T01:56:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:06 smithi103 ceph-mon[30902]: pgmap v98: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T01:56:08.594 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:08 smithi139 ceph-mon[34262]: pgmap v99: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-05T01:56:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:08 smithi167 ceph-mon[35122]: pgmap v99: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-05T01:56:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:08 smithi103 ceph-mon[30902]: pgmap v99: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-05T01:56:10.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:10 smithi139 ceph-mon[34262]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T01:56:10.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T01:56:10.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:10 smithi167 ceph-mon[35122]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T01:56:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T01:56:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:10 smithi103 ceph-mon[30902]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T01:56:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T01:56:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:11.446 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:11 smithi139 ceph-mon[34262]: Deploying daemon osd.4 on smithi139 2024-09-05T01:56:11.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:11 smithi167 ceph-mon[35122]: Deploying daemon osd.4 on smithi139 2024-09-05T01:56:11.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:11 smithi103 ceph-mon[30902]: Deploying daemon osd.4 on smithi139 2024-09-05T01:56:12.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:12 smithi139 ceph-mon[34262]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:12.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:12 smithi167 ceph-mon[35122]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:12 smithi103 ceph-mon[30902]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:14.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:14 smithi139 ceph-mon[34262]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:14 smithi167 ceph-mon[35122]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:14 smithi103 ceph-mon[30902]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:16.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:16 smithi139 ceph-mon[34262]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:16.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:16.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:16.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:16 smithi167 ceph-mon[35122]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:16 smithi103 ceph-mon[30902]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:16.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:18.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:18 smithi139 ceph-mon[34262]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:18.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:18.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:18.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:18.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:18.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:18 smithi167 ceph-mon[35122]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:18.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:18.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:18 smithi103 ceph-mon[30902]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:18.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:18.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:19.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.517+0000 7f605cff9640 1 -- 172.21.15.139:0/2392451724 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6060062630 con 0x7f60480a2bb0 2024-09-05T01:56:19.519 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 4 on host 'smithi139' 2024-09-05T01:56:19.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60480a2bb0 msgr2=0x7f60480a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:19.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60480a2bb0 0x7f60480a5070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6054000c00 tx=0x7f60540314a0 comp rx=0 tx=0).stop 2024-09-05T01:56:19.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060102720 msgr2=0x7f6060197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:19.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060102720 0x7f6060197e90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f605000e9f0 tx=0x7f605000edf0 comp rx=0 tx=0).stop 2024-09-05T01:56:19.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 shutdown_connections 2024-09-05T01:56:19.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6060107980 0x7f606019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:19.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60480a2bb0 0x7f60480a5070 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:19.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6060105050 0x7f60601983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:19.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 --2- 172.21.15.139:0/2392451724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6060102720 0x7f6060197e90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:19.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 >> 172.21.15.139:0/2392451724 conn(0x7f60600fc2a0 msgr2=0x7f6060102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:56:19.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 shutdown_connections 2024-09-05T01:56:19.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:56:19.520+0000 7f6065f8f640 1 -- 172.21.15.139:0/2392451724 wait complete. 2024-09-05T01:56:20.211 DEBUG:teuthology.orchestra.run.smithi139:osd.4> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.4.service 2024-09-05T01:56:20.213 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-09-05T01:56:20.213 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T01:56:20.464 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:56:20.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:20 smithi139 ceph-mon[34262]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:20.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:20.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:20.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:20.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:20 smithi167 ceph-mon[35122]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:20.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:20.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:20.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:20 smithi103 ceph-mon[30902]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:20.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:21.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:21 smithi139 ceph-mon[34262]: from='osd.4 [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:56:21.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:21 smithi139 ceph-mon[34262]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:56:21.396 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 01:56:21 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4[45552]: 2024-09-05T01:56:21.022+0000 7fe6a8a39640 -1 osd.4 0 log_to_monitors true 2024-09-05T01:56:21.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:21 smithi167 ceph-mon[35122]: from='osd.4 [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:56:21.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:21 smithi167 ceph-mon[35122]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:56:21.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:21 smithi103 ceph-mon[30902]: from='osd.4 [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:56:21.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:21 smithi103 ceph-mon[30902]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:56:22.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:22 smithi167 ceph-mon[35122]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:22.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:22 smithi167 ceph-mon[35122]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T01:56:22.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:22 smithi167 ceph-mon[35122]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T01:56:22.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:22 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:22.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:22 smithi167 ceph-mon[35122]: from='osd.4 [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:56:22.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:22 smithi167 ceph-mon[35122]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:56:22.517 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:22 smithi139 ceph-mon[34262]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:22.517 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:22 smithi139 ceph-mon[34262]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T01:56:22.517 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:22 smithi139 ceph-mon[34262]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T01:56:22.517 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:22 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:22.517 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:22 smithi139 ceph-mon[34262]: from='osd.4 [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:56:22.517 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:22 smithi139 ceph-mon[34262]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:56:22.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:22 smithi103 ceph-mon[30902]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:22.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:22 smithi103 ceph-mon[30902]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T01:56:22.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:22 smithi103 ceph-mon[30902]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T01:56:22.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:22 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:22.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:22 smithi103 ceph-mon[30902]: from='osd.4 [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:56:22.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:22 smithi103 ceph-mon[30902]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:56:23.220 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 01:56:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4[45552]: 2024-09-05T01:56:22.898+0000 7fe6a49bc640 -1 osd.4 0 waiting for initial osdmap 2024-09-05T01:56:23.220 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 01:56:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4[45552]: 2024-09-05T01:56:22.904+0000 7fe69c7dc640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:56:23.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:23 smithi167 ceph-mon[35122]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:56:23.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:23 smithi167 ceph-mon[35122]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T01:56:23.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:23.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:23.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:23 smithi167 ceph-mon[35122]: from='osd.4 ' entity='osd.4' 2024-09-05T01:56:23.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:23.365 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:56:23.390 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-09-05T01:56:23.641 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:56:23.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:23 smithi139 ceph-mon[34262]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:56:23.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:23 smithi139 ceph-mon[34262]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T01:56:23.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:23.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:23.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:23 smithi139 ceph-mon[34262]: from='osd.4 ' entity='osd.4' 2024-09-05T01:56:23.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:23.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:23 smithi103 ceph-mon[30902]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:56:23.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:23 smithi103 ceph-mon[30902]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T01:56:23.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:23.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:23.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:23 smithi103 ceph-mon[30902]: from='osd.4 ' entity='osd.4' 2024-09-05T01:56:23.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 -- 172.21.15.167:0/1360618888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec1026c0 msgr2=0x7fa1ec108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/1360618888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec1026c0 0x7fa1ec108f50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009ca0 tx=0x7fa1e402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 -- 172.21.15.167:0/1360618888 shutdown_connections 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/1360618888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec1026c0 0x7fa1ec108f50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/1360618888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1ec101d00 0x7fa1ec102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/1360618888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ec100b00 0x7fa1ec100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 -- 172.21.15.167:0/1360618888 >> 172.21.15.167:0/1360618888 conn(0x7fa1ec0fc290 msgr2=0x7fa1ec0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 -- 172.21.15.167:0/1360618888 shutdown_connections 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 -- 172.21.15.167:0/1360618888 wait complete. 2024-09-05T01:56:24.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.093+0000 7fa1f16f2640 1 Processor -- start 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1f16f2640 1 -- start start 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1f16f2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec100b00 0x7fa1ec104be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1f16f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ec101d00 0x7fa1ec103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1f16f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1ec1026c0 0x7fa1ec1037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1f16f2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10c630 con 0x7fa1ec1026c0 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1f16f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10c4b0 con 0x7fa1ec101d00 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1f16f2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10c7b0 con 0x7fa1ec100b00 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec100b00 0x7fa1ec104be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec100b00 0x7fa1ec104be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59970/0 (socket says 172.21.15.167:59970) 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eaffd640 1 -- 172.21.15.167:0/3641777102 learned_addr learned my addr 172.21.15.167:0/3641777102 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1ea7fc640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ec101d00 0x7fa1ec103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:56:24.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eb7fe640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1ec1026c0 0x7fa1ec1037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eaffd640 1 -- 172.21.15.167:0/3641777102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ec101d00 msgr2=0x7fa1ec103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eaffd640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ec101d00 0x7fa1ec103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eaffd640 1 -- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1ec1026c0 msgr2=0x7fa1ec1037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eaffd640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1ec1026c0 0x7fa1ec1037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eaffd640 1 -- 172.21.15.167:0/3641777102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec103ff0 con 0x7fa1ec100b00 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eb7fe640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1ec1026c0 0x7fa1ec1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.094+0000 7fa1eaffd640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec100b00 0x7fa1ec104be0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000d950 tx=0x7fa1e000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.095+0000 7fa1d3fff640 1 -- 172.21.15.167:0/3641777102 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0017070 con 0x7fa1ec100b00 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.095+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec1a1ee0 con 0x7fa1ec100b00 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.095+0000 7fa1d3fff640 1 -- 172.21.15.167:0/3641777102 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1e0009c30 con 0x7fa1ec100b00 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.095+0000 7fa1d3fff640 1 -- 172.21.15.167:0/3641777102 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000f620 con 0x7fa1ec100b00 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.095+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec1a22d0 con 0x7fa1ec100b00 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.096+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec1a25c0 con 0x7fa1ec100b00 2024-09-05T01:56:24.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.097+0000 7fa1d3fff640 1 -- 172.21.15.167:0/3641777102 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa1e001a070 con 0x7fa1ec100b00 2024-09-05T01:56:24.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.097+0000 7fa1d3fff640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1d40a2b80 0x7fa1d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:56:24.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.098+0000 7fa1d3fff640 1 -- 172.21.15.167:0/3641777102 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4101+0+0 (secure 0 0 0) 0x7fa1e00cd9d0 con 0x7fa1ec100b00 2024-09-05T01:56:24.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.098+0000 7fa1ea7fc640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1d40a2b80 0x7fa1d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:56:24.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.098+0000 7fa1ea7fc640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1d40a2b80 0x7fa1d40a5040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec104960 tx=0x7fa1dc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:56:24.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.102+0000 7fa1d3fff640 1 -- 172.21.15.167:0/3641777102 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1e0009de0 con 0x7fa1ec100b00 2024-09-05T01:56:24.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:24.285+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa1ec1a2790 con 0x7fa1d40a2b80 2024-09-05T01:56:24.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:56:24.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:56:24.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:24.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:24.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:24.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:24.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:24.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: osd.4 [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] boot 2024-09-05T01:56:24.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: osdmap e31: 5 total, 5 up, 5 in 2024-09-05T01:56:24.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:56:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:56:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:24.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:24.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: osd.4 [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] boot 2024-09-05T01:56:24.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: osdmap e31: 5 total, 5 up, 5 in 2024-09-05T01:56:24.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:56:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:56:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:56:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: osd.4 [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] boot 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: osdmap e31: 5 total, 5 up, 5 in 2024-09-05T01:56:24.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:56:25.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:25 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi139 2024-09-05T01:56:25.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:25 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi139 to 8544M 2024-09-05T01:56:25.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:56:25.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:56:25.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:25.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:25 smithi167 ceph-mon[35122]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T01:56:25.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:25 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi139 2024-09-05T01:56:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:25 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi139 to 8544M 2024-09-05T01:56:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:56:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:56:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:25 smithi139 ceph-mon[34262]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T01:56:25.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:25 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi139 2024-09-05T01:56:25.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:25 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi139 to 8544M 2024-09-05T01:56:25.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:56:25.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:56:25.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:25.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:25 smithi103 ceph-mon[30902]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T01:56:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:26 smithi167 ceph-mon[35122]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:56:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:26 smithi167 ceph-mon[35122]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:26.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:26 smithi139 ceph-mon[34262]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:56:26.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:26 smithi139 ceph-mon[34262]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:26.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:26 smithi103 ceph-mon[30902]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:56:26.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:26 smithi103 ceph-mon[30902]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:27 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/2903864892' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37eca6d4-b597-41fc-af54-5db67c9a4d57"}]: dispatch 2024-09-05T01:56:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:27 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37eca6d4-b597-41fc-af54-5db67c9a4d57"}]: dispatch 2024-09-05T01:56:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:27 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37eca6d4-b597-41fc-af54-5db67c9a4d57"}]': finished 2024-09-05T01:56:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:27 smithi167 ceph-mon[35122]: osdmap e33: 6 total, 5 up, 6 in 2024-09-05T01:56:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:27 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/1385531121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:56:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:27 smithi139 ceph-mon[34262]: from='client.? 172.21.15.167:0/2903864892' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37eca6d4-b597-41fc-af54-5db67c9a4d57"}]: dispatch 2024-09-05T01:56:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:27 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37eca6d4-b597-41fc-af54-5db67c9a4d57"}]: dispatch 2024-09-05T01:56:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:27 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37eca6d4-b597-41fc-af54-5db67c9a4d57"}]': finished 2024-09-05T01:56:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:27 smithi139 ceph-mon[34262]: osdmap e33: 6 total, 5 up, 6 in 2024-09-05T01:56:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:27 smithi139 ceph-mon[34262]: from='client.? 172.21.15.167:0/1385531121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:56:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:27 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/2903864892' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37eca6d4-b597-41fc-af54-5db67c9a4d57"}]: dispatch 2024-09-05T01:56:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:27 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37eca6d4-b597-41fc-af54-5db67c9a4d57"}]: dispatch 2024-09-05T01:56:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:27 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37eca6d4-b597-41fc-af54-5db67c9a4d57"}]': finished 2024-09-05T01:56:27.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:27 smithi103 ceph-mon[30902]: osdmap e33: 6 total, 5 up, 6 in 2024-09-05T01:56:27.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:27.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:27 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/1385531121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:56:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:28 smithi167 ceph-mon[35122]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:28.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:28 smithi139 ceph-mon[34262]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:28 smithi103 ceph-mon[30902]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:30.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:30 smithi167 ceph-mon[35122]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:30.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:30 smithi139 ceph-mon[34262]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:30 smithi103 ceph-mon[30902]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:32.617 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:32 smithi167 ceph-mon[35122]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:32.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:32 smithi139 ceph-mon[34262]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:32.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:32 smithi103 ceph-mon[30902]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:34 smithi167 ceph-mon[35122]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:34.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:34 smithi139 ceph-mon[34262]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:34.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:34 smithi103 ceph-mon[30902]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:35.515 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:35 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T01:56:35.515 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:35 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:35.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:35 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T01:56:35.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:35 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:35.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:35 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T01:56:35.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:35 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:36.517 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:36 smithi167 ceph-mon[35122]: Deploying daemon osd.5 on smithi167 2024-09-05T01:56:36.517 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:36 smithi167 ceph-mon[35122]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:36.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:36 smithi139 ceph-mon[34262]: Deploying daemon osd.5 on smithi167 2024-09-05T01:56:36.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:36 smithi139 ceph-mon[34262]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:36 smithi103 ceph-mon[30902]: Deploying daemon osd.5 on smithi167 2024-09-05T01:56:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:36 smithi103 ceph-mon[30902]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:38.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:38 smithi167 ceph-mon[35122]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:38.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:38 smithi139 ceph-mon[34262]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:38 smithi103 ceph-mon[30902]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:39.518 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:39 smithi167 ceph-mon[35122]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:39.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:39 smithi139 ceph-mon[34262]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:39.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:39 smithi103 ceph-mon[30902]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:40.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:40.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:40.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:40.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:40.785 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:40.785 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:40.786 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:40.786 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:40.786 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:41.617 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:41 smithi167 ceph-mon[35122]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:41.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:41 smithi139 ceph-mon[34262]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:41.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:41 smithi103 ceph-mon[30902]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:43.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.448+0000 7fa1d3fff640 1 -- 172.21.15.167:0/3641777102 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa1ec1a2790 con 0x7fa1d40a2b80 2024-09-05T01:56:43.450 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-09-05T01:56:43.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1d40a2b80 msgr2=0x7fa1d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:43.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1d40a2b80 0x7fa1d40a5040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec104960 tx=0x7fa1dc00b410 comp rx=0 tx=0).stop 2024-09-05T01:56:43.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec100b00 msgr2=0x7fa1ec104be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:43.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec100b00 0x7fa1ec104be0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000d950 tx=0x7fa1e000de10 comp rx=0 tx=0).stop 2024-09-05T01:56:43.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 shutdown_connections 2024-09-05T01:56:43.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1ec1026c0 0x7fa1ec1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:43.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ec101d00 0x7fa1ec103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:43.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1d40a2b80 0x7fa1d40a5040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:43.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 --2- 172.21.15.167:0/3641777102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ec100b00 0x7fa1ec104be0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:43.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 >> 172.21.15.167:0/3641777102 conn(0x7fa1ec0fc290 msgr2=0x7fa1ec06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:56:43.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 shutdown_connections 2024-09-05T01:56:43.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:56:43.451+0000 7fa1f16f2640 1 -- 172.21.15.167:0/3641777102 wait complete. 2024-09-05T01:56:44.046 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.5.service 2024-09-05T01:56:44.049 INFO:tasks.cephadm:Deploying osd.6 on smithi103 with /dev/vg_nvme/lv_2... 2024-09-05T01:56:44.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T01:56:44.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:44 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:44 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:44 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:44.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:44 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:44.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:44 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:44 smithi167 ceph-mon[35122]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:44.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:44 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:44.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:44 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:44 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.307 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:56:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:44 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:44 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:44 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:44 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:44 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:44 smithi139 ceph-mon[34262]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:44 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:44 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:44 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:44 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:44 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:44 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:44 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:44 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:44 smithi103 ceph-mon[30902]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:44 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:56:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:44 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:44.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:44 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:46.346 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:46 smithi103 ceph-mon[30902]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:46.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:46 smithi167 ceph-mon[35122]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:46.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:46 smithi139 ceph-mon[34262]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:46.636 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 01:56:46 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T01:56:46.451+0000 7feb3855a640 -1 osd.5 0 log_to_monitors true 2024-09-05T01:56:46.871 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:56:46.896 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-09-05T01:56:47.146 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:56:47.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:47 smithi103 ceph-mon[30902]: from='osd.5 [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:56:47.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:47 smithi103 ceph-mon[30902]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:56:47.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:47 smithi167 ceph-mon[35122]: from='osd.5 [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:56:47.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:47 smithi167 ceph-mon[35122]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:56:47.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:47 smithi139 ceph-mon[34262]: from='osd.5 [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:56:47.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:47 smithi139 ceph-mon[34262]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:56:47.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.637+0000 7f96d4793640 1 -- 172.21.15.103:0/2709117745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc1037c0 msgr2=0x7f96cc10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:47.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.637+0000 7f96d4793640 1 --2- 172.21.15.103:0/2709117745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc1037c0 0x7f96cc10a050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009ca0 tx=0x7f96c402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:56:47.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.637+0000 7f96d4793640 1 -- 172.21.15.103:0/2709117745 shutdown_connections 2024-09-05T01:56:47.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.637+0000 7f96d4793640 1 --2- 172.21.15.103:0/2709117745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc1037c0 0x7f96cc10a050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.637+0000 7f96d4793640 1 --2- 172.21.15.103:0/2709117745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96cc102e00 0x7f96cc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.637+0000 7f96d4793640 1 --2- 172.21.15.103:0/2709117745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc101c00 0x7f96cc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.637+0000 7f96d4793640 1 -- 172.21.15.103:0/2709117745 >> 172.21.15.103:0/2709117745 conn(0x7f96cc0fd390 msgr2=0x7f96cc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.637+0000 7f96d4793640 1 -- 172.21.15.103:0/2709117745 shutdown_connections 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.637+0000 7f96d4793640 1 -- 172.21.15.103:0/2709117745 wait complete. 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d4793640 1 Processor -- start 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d4793640 1 -- start start 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d4793640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc101c00 0x7f96cc075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d4793640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc102e00 0x7f96cc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d4793640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96cc1037c0 0x7f96cc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d4793640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc10d710 con 0x7f96cc102e00 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d4793640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc10d590 con 0x7f96cc101c00 2024-09-05T01:56:47.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d4793640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc10d890 con 0x7f96cc1037c0 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d1d2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc102e00 0x7f96cc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d1d2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc102e00 0x7f96cc0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35564/0 (socket says 172.21.15.103:35564) 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d1d2e640 1 -- 172.21.15.103:0/2809318571 learned_addr learned my addr 172.21.15.103:0/2809318571 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d2d30640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96cc1037c0 0x7f96cc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d252f640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc101c00 0x7f96cc075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d1d2e640 1 -- 172.21.15.103:0/2809318571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96cc1037c0 msgr2=0x7f96cc074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d1d2e640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96cc1037c0 0x7f96cc074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d1d2e640 1 -- 172.21.15.103:0/2809318571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc101c00 msgr2=0x7f96cc075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d1d2e640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc101c00 0x7f96cc075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d1d2e640 1 -- 172.21.15.103:0/2809318571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc0753e0 con 0x7f96cc102e00 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.638+0000 7f96d252f640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc101c00 0x7f96cc075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:56:47.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.639+0000 7f96d1d2e640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc102e00 0x7f96cc0745e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96c800d950 tx=0x7f96c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.639+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2809318571 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8017070 con 0x7f96cc102e00 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.639+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2809318571 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c8009c30 con 0x7f96cc102e00 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.639+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1a1ec0 con 0x7f96cc102e00 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.639+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2809318571 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c800f620 con 0x7f96cc102e00 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.639+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc1a2370 con 0x7f96cc102e00 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.640+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc109000 con 0x7f96cc102e00 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.641+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2809318571 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f96c80109a0 con 0x7f96cc102e00 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.641+0000 7f96bf7fe640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f96b80a2c00 0x7f96b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.642+0000 7f96d252f640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f96b80a2c00 0x7f96b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.642+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2809318571 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4237+0+0 (secure 0 0 0) 0x7f96c80cdc10 con 0x7f96cc102e00 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.642+0000 7f96d252f640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f96b80a2c00 0x7f96b80a50c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96cc102c60 tx=0x7f96c000a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:56:47.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.645+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2809318571 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96c8009de0 con 0x7f96cc102e00 2024-09-05T01:56:47.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:56:47.818+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f96cc062630 con 0x7f96b80a2c00 2024-09-05T01:56:48.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:48.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T01:56:48.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='osd.5 [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:56:48.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T01:56:48.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:48.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:56:48.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi167 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi167 to 8544M 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:56:48.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:56:48.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:48.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:48.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T01:56:48.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='osd.5 [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:56:48.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi167 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi167 to 8544M 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:48.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:48.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:56:48.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:56:48.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='osd.5 [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi167 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:48.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:56:48.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi167 to 8544M 2024-09-05T01:56:48.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:48.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:56:48.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:56:48.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:56:48.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:56:48.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:49.110 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T01:56:48.771+0000 7feb344dd640 -1 osd.5 0 waiting for initial osdmap 2024-09-05T01:56:49.110 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 01:56:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T01:56:48.777+0000 7feb2c2fd640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:56:49.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:49 smithi103 ceph-mon[30902]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:56:49.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:49 smithi103 ceph-mon[30902]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:56:49.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:49 smithi103 ceph-mon[30902]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T01:56:49.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:49 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:49.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:49 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:49.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:49 smithi103 ceph-mon[30902]: from='osd.5 ' entity='osd.5' 2024-09-05T01:56:49.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:49 smithi167 ceph-mon[35122]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:56:49.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:49 smithi167 ceph-mon[35122]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:56:49.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:49 smithi167 ceph-mon[35122]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T01:56:49.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:49 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:49.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:49 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:49.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:49 smithi167 ceph-mon[35122]: from='osd.5 ' entity='osd.5' 2024-09-05T01:56:49.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:49 smithi139 ceph-mon[34262]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:56:49.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:49 smithi139 ceph-mon[34262]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:56:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:49 smithi139 ceph-mon[34262]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T01:56:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:49 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:49 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:49.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:49 smithi139 ceph-mon[34262]: from='osd.5 ' entity='osd.5' 2024-09-05T01:56:50.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:50 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:56:50.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:50 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:56:50.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:50 smithi103 ceph-mon[30902]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:50.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:50 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:50.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:50 smithi103 ceph-mon[30902]: osd.5 [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] boot 2024-09-05T01:56:50.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:50 smithi103 ceph-mon[30902]: osdmap e36: 6 total, 6 up, 6 in 2024-09-05T01:56:50.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:50 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:50.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:50 smithi103 ceph-mon[30902]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T01:56:50.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:50 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:56:50.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:50 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:56:50.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:50 smithi167 ceph-mon[35122]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:50.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:50 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:50.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:50 smithi167 ceph-mon[35122]: osd.5 [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] boot 2024-09-05T01:56:50.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:50 smithi167 ceph-mon[35122]: osdmap e36: 6 total, 6 up, 6 in 2024-09-05T01:56:50.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:50 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:50.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:50 smithi167 ceph-mon[35122]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T01:56:50.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:50 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:56:50.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:50 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:56:50.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:50 smithi139 ceph-mon[34262]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:56:50.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:50 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:50.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:50 smithi139 ceph-mon[34262]: osd.5 [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] boot 2024-09-05T01:56:50.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:50 smithi139 ceph-mon[34262]: osdmap e36: 6 total, 6 up, 6 in 2024-09-05T01:56:50.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:50 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:56:50.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:50 smithi139 ceph-mon[34262]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T01:56:51.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:51 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/150006062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82343e71-6c0c-4ea3-b050-23b2bcae9e50"}]: dispatch 2024-09-05T01:56:51.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:51 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/150006062' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82343e71-6c0c-4ea3-b050-23b2bcae9e50"}]': finished 2024-09-05T01:56:51.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:51 smithi167 ceph-mon[35122]: osdmap e38: 7 total, 6 up, 7 in 2024-09-05T01:56:51.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:51 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:56:51.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:51 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/898131297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:56:51.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:51 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/150006062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82343e71-6c0c-4ea3-b050-23b2bcae9e50"}]: dispatch 2024-09-05T01:56:51.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:51 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/150006062' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82343e71-6c0c-4ea3-b050-23b2bcae9e50"}]': finished 2024-09-05T01:56:51.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:51 smithi139 ceph-mon[34262]: osdmap e38: 7 total, 6 up, 7 in 2024-09-05T01:56:51.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:51 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:56:51.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:51 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/898131297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:56:51.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:51 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/150006062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82343e71-6c0c-4ea3-b050-23b2bcae9e50"}]: dispatch 2024-09-05T01:56:51.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:51 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/150006062' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82343e71-6c0c-4ea3-b050-23b2bcae9e50"}]': finished 2024-09-05T01:56:51.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:51 smithi103 ceph-mon[30902]: osdmap e38: 7 total, 6 up, 7 in 2024-09-05T01:56:51.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:51 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:56:51.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:51 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/898131297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:56:52.538 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:52 smithi103 ceph-mon[30902]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:56:52.538 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:52 smithi103 ceph-mon[30902]: osdmap e39: 7 total, 6 up, 7 in 2024-09-05T01:56:52.538 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:56:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:52 smithi167 ceph-mon[35122]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:56:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:52 smithi167 ceph-mon[35122]: osdmap e39: 7 total, 6 up, 7 in 2024-09-05T01:56:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:56:52.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:52 smithi139 ceph-mon[34262]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:56:52.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:52 smithi139 ceph-mon[34262]: osdmap e39: 7 total, 6 up, 7 in 2024-09-05T01:56:52.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:56:53.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:53 smithi167 ceph-mon[35122]: osdmap e40: 7 total, 6 up, 7 in 2024-09-05T01:56:53.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:56:53.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:53 smithi139 ceph-mon[34262]: osdmap e40: 7 total, 6 up, 7 in 2024-09-05T01:56:53.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:56:53.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:53 smithi103 ceph-mon[30902]: osdmap e40: 7 total, 6 up, 7 in 2024-09-05T01:56:53.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:56:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:54 smithi167 ceph-mon[35122]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-05T01:56:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:54 smithi139 ceph-mon[34262]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-05T01:56:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:54 smithi103 ceph-mon[30902]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-05T01:56:56.479 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:56 smithi103 ceph-mon[30902]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-05T01:56:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:56 smithi167 ceph-mon[35122]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-05T01:56:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:56 smithi139 ceph-mon[34262]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-05T01:56:58.546 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:58 smithi103 ceph-mon[30902]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-05T01:56:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:58 smithi167 ceph-mon[35122]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-05T01:56:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:58 smithi139 ceph-mon[34262]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-05T01:56:59.316 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T01:56:59.316 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:56:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T01:56:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:56:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:56:59.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T01:56:59.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:56:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:00.337 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:00 smithi103 ceph-mon[30902]: Deploying daemon osd.6 on smithi103 2024-09-05T01:57:00.337 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:00 smithi103 ceph-mon[30902]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T01:57:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:00 smithi167 ceph-mon[35122]: Deploying daemon osd.6 on smithi103 2024-09-05T01:57:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:00 smithi167 ceph-mon[35122]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T01:57:00.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:00 smithi139 ceph-mon[34262]: Deploying daemon osd.6 on smithi103 2024-09-05T01:57:00.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:00 smithi139 ceph-mon[34262]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T01:57:02.375 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:02 smithi103 ceph-mon[30902]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-05T01:57:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:02 smithi167 ceph-mon[35122]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-05T01:57:02.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:02 smithi139 ceph-mon[34262]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-05T01:57:04.401 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:04 smithi103 ceph-mon[30902]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:04.402 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:04.402 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:04.402 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:04 smithi167 ceph-mon[35122]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:04.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:04.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:04 smithi139 ceph-mon[34262]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:04.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:04.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:04.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:06.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:06 smithi103 ceph-mon[30902]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:06 smithi167 ceph-mon[35122]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:06.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:06 smithi139 ceph-mon[34262]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:07.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.510+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2809318571 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f96cc062630 con 0x7f96b80a2c00 2024-09-05T01:57:07.511 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 6 on host 'smithi103' 2024-09-05T01:57:07.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f96b80a2c00 msgr2=0x7f96b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:07.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f96b80a2c00 0x7f96b80a50c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96cc102c60 tx=0x7f96c000a5e0 comp rx=0 tx=0).stop 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc102e00 msgr2=0x7f96cc0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc102e00 0x7f96cc0745e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96c800d950 tx=0x7f96c800de10 comp rx=0 tx=0).stop 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 shutdown_connections 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f96b80a2c00 0x7f96b80a50c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96cc1037c0 0x7f96cc074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc102e00 0x7f96cc0745e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 --2- 172.21.15.103:0/2809318571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc101c00 0x7f96cc075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 >> 172.21.15.103:0/2809318571 conn(0x7f96cc0fd390 msgr2=0x7f96cc1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 shutdown_connections 2024-09-05T01:57:07.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:57:07.513+0000 7f96d4793640 1 -- 172.21.15.103:0/2809318571 wait complete. 2024-09-05T01:57:08.200 DEBUG:teuthology.orchestra.run.smithi103:osd.6> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.6.service 2024-09-05T01:57:08.203 INFO:tasks.cephadm:Deploying osd.7 on smithi139 with /dev/vg_nvme/lv_2... 2024-09-05T01:57:08.203 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T01:57:08.456 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:57:08.476 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:08 smithi103 ceph-mon[30902]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:08.476 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:08 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.476 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:08 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.476 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:08 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.477 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:08 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:08 smithi139 ceph-mon[34262]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:08.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:08 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:08 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:08 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:08 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:08 smithi167 ceph-mon[35122]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:08 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:08 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:08 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:08 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:10.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:10 smithi139 ceph-mon[34262]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:10.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:10 smithi139 ceph-mon[34262]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T01:57:10.404 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:10 smithi103 ceph-mon[30902]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:10.405 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:10 smithi103 ceph-mon[30902]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T01:57:10.405 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 01:57:10 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T01:57:10.153+0000 7f3f0cc3d640 -1 osd.6 0 log_to_monitors true 2024-09-05T01:57:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:10 smithi167 ceph-mon[35122]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:10 smithi167 ceph-mon[35122]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T01:57:11.194 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:57:11.219 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_2 2024-09-05T01:57:11.398 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T01:57:11.256+0000 7f3f08bc0640 -1 osd.6 0 waiting for initial osdmap 2024-09-05T01:57:11.418 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T01:57:11.267+0000 7f3f049e8640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:57:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T01:57:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T01:57:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:57:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi103 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:57:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T01:57:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:11.481 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi103 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:11.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:57:11.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T01:57:11.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:11.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi103 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-05T01:57:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T01:57:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.000+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2745265848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 msgr2=0x7ff7f4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.000+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2745265848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 0x7ff7f4102000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009900 tx=0x7ff7e402a410 comp rx=0 tx=0).stop 2024-09-05T01:57:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.000+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2745265848 shutdown_connections 2024-09-05T01:57:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.000+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2745265848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f41037c0 0x7ff7f410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.000+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2745265848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7f4102e00 0x7ff7f4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.000+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2745265848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 0x7ff7f4102000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.000+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2745265848 >> 172.21.15.139:0/2745265848 conn(0x7ff7f40fd390 msgr2=0x7ff7f40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:57:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.000+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2745265848 shutdown_connections 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.000+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2745265848 wait complete. 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f8e74640 1 Processor -- start 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f8e74640 1 -- start start 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f8e74640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 0x7ff7f4197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f8e74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7f4102e00 0x7ff7f41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f8e74640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f41037c0 0x7ff7f419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f8e74640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f410d710 con 0x7ff7f4102e00 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f8e74640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f410d590 con 0x7ff7f4101c00 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f8e74640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f410d890 con 0x7ff7f41037c0 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 0x7ff7f4197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 0x7ff7f4197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56722/0 (socket says 172.21.15.139:56722) 2024-09-05T01:57:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f259c640 1 -- 172.21.15.139:0/2019700747 learned_addr learned my addr 172.21.15.139:0/2019700747 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f2d9d640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f41037c0 0x7ff7f419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f1d9b640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7f4102e00 0x7ff7f41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f259c640 1 -- 172.21.15.139:0/2019700747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f41037c0 msgr2=0x7ff7f419f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f259c640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f41037c0 0x7ff7f419f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f259c640 1 -- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7f4102e00 msgr2=0x7ff7f41982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f259c640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7f4102e00 0x7ff7f41982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f259c640 1 -- 172.21.15.139:0/2019700747 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f419f990 con 0x7ff7f4101c00 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7f259c640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 0x7ff7f4197d60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009230 tx=0x7ff7e4004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.001+0000 7ff7e37fe640 1 -- 172.21.15.139:0/2019700747 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e403f030 con 0x7ff7f4101c00 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.002+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f419fc20 con 0x7ff7f4101c00 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.002+0000 7ff7e37fe640 1 -- 172.21.15.139:0/2019700747 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7e4003cb0 con 0x7ff7f4101c00 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.002+0000 7ff7e37fe640 1 -- 172.21.15.139:0/2019700747 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4035070 con 0x7ff7f4101c00 2024-09-05T01:57:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.002+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f41a0150 con 0x7ff7f4101c00 2024-09-05T01:57:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.003+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f4109000 con 0x7ff7f4101c00 2024-09-05T01:57:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.004+0000 7ff7e37fe640 1 -- 172.21.15.139:0/2019700747 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff7e4042030 con 0x7ff7f4101c00 2024-09-05T01:57:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.004+0000 7ff7e37fe640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff7dc0a2bb0 0x7ff7dc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:57:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.004+0000 7ff7e37fe640 1 -- 172.21.15.139:0/2019700747 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4685+0+0 (secure 0 0 0) 0x7ff7e40ee940 con 0x7ff7f4101c00 2024-09-05T01:57:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.005+0000 7ff7f1d9b640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff7dc0a2bb0 0x7ff7dc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:57:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.005+0000 7ff7f1d9b640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff7dc0a2bb0 0x7ff7dc0a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4199280 tx=0x7ff7e8005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:57:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.009+0000 7ff7e37fe640 1 -- 172.21.15.139:0/2019700747 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7e40bc6d0 con 0x7ff7f4101c00 2024-09-05T01:57:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:12.182+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff7f4062630 con 0x7ff7dc0a2bb0 2024-09-05T01:57:12.591 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:12 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:12.592 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:12 smithi139 ceph-mon[34262]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:57:12.592 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:12 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:57:12.592 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:12 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:57:12.592 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:12 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:12.592 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:12 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:12 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:12 smithi167 ceph-mon[35122]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:57:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:12 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:57:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:12 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:57:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:12 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:12.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:12 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:12 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:12 smithi103 ceph-mon[30902]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:57:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:12 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:57:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:12 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:57:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:12 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:12 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:13.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:13 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:57:13.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:13 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:57:13.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:13 smithi167 ceph-mon[35122]: osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] boot 2024-09-05T01:57:13.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:13 smithi167 ceph-mon[35122]: osdmap e43: 7 total, 7 up, 7 in 2024-09-05T01:57:13.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:13.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:13 smithi167 ceph-mon[35122]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:13.642 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:13 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:57:13.642 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:13 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:57:13.642 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:13 smithi139 ceph-mon[34262]: osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] boot 2024-09-05T01:57:13.642 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:13 smithi139 ceph-mon[34262]: osdmap e43: 7 total, 7 up, 7 in 2024-09-05T01:57:13.643 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:13.643 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:13 smithi139 ceph-mon[34262]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:13 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:57:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:13 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:57:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:13 smithi103 ceph-mon[30902]: osd.6 [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] boot 2024-09-05T01:57:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:13 smithi103 ceph-mon[30902]: osdmap e43: 7 total, 7 up, 7 in 2024-09-05T01:57:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:57:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:13 smithi103 ceph-mon[30902]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:57:14.272 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:14 smithi139 ceph-mon[34262]: osdmap e44: 7 total, 7 up, 7 in 2024-09-05T01:57:14.606 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:14 smithi103 ceph-mon[30902]: osdmap e44: 7 total, 7 up, 7 in 2024-09-05T01:57:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:14 smithi167 ceph-mon[35122]: osdmap e44: 7 total, 7 up, 7 in 2024-09-05T01:57:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:15 smithi167 ceph-mon[35122]: from='client.? 172.21.15.139:0/3286165670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c96851b2-9ad0-4adb-b840-9bd11ab6655e"}]: dispatch 2024-09-05T01:57:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:15 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c96851b2-9ad0-4adb-b840-9bd11ab6655e"}]: dispatch 2024-09-05T01:57:15.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:15 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c96851b2-9ad0-4adb-b840-9bd11ab6655e"}]': finished 2024-09-05T01:57:15.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:15 smithi167 ceph-mon[35122]: osdmap e45: 8 total, 7 up, 8 in 2024-09-05T01:57:15.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:15.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:15 smithi167 ceph-mon[35122]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:15.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:15 smithi167 ceph-mon[35122]: from='client.? 172.21.15.139:0/995285264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:57:15.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:15 smithi139 ceph-mon[34262]: from='client.? 172.21.15.139:0/3286165670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c96851b2-9ad0-4adb-b840-9bd11ab6655e"}]: dispatch 2024-09-05T01:57:15.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:15 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c96851b2-9ad0-4adb-b840-9bd11ab6655e"}]: dispatch 2024-09-05T01:57:15.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:15 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c96851b2-9ad0-4adb-b840-9bd11ab6655e"}]': finished 2024-09-05T01:57:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:15 smithi139 ceph-mon[34262]: osdmap e45: 8 total, 7 up, 8 in 2024-09-05T01:57:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:15 smithi139 ceph-mon[34262]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:15 smithi139 ceph-mon[34262]: from='client.? 172.21.15.139:0/995285264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:57:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:15 smithi103 ceph-mon[30902]: from='client.? 172.21.15.139:0/3286165670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c96851b2-9ad0-4adb-b840-9bd11ab6655e"}]: dispatch 2024-09-05T01:57:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:15 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c96851b2-9ad0-4adb-b840-9bd11ab6655e"}]: dispatch 2024-09-05T01:57:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:15 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c96851b2-9ad0-4adb-b840-9bd11ab6655e"}]': finished 2024-09-05T01:57:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:15 smithi103 ceph-mon[30902]: osdmap e45: 8 total, 7 up, 8 in 2024-09-05T01:57:15.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:15.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:15 smithi103 ceph-mon[30902]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:15.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:15 smithi103 ceph-mon[30902]: from='client.? 172.21.15.139:0/995285264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:57:18.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:18 smithi167 ceph-mon[35122]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:18.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:18 smithi139 ceph-mon[34262]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:18.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:18 smithi103 ceph-mon[30902]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:20.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:20 smithi167 ceph-mon[35122]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:20.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:20 smithi139 ceph-mon[34262]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:20.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:20 smithi103 ceph-mon[30902]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:22.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:22 smithi139 ceph-mon[34262]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:22.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:22 smithi167 ceph-mon[35122]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:22.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:22 smithi103 ceph-mon[30902]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:24.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:24 smithi167 ceph-mon[35122]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:24.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T01:57:24.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:24 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:24.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:24 smithi139 ceph-mon[34262]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T01:57:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:24 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:24.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:24 smithi103 ceph-mon[30902]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:24.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T01:57:24.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:24 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:25.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:25 smithi167 ceph-mon[35122]: Deploying daemon osd.7 on smithi139 2024-09-05T01:57:25.393 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:25 smithi139 ceph-mon[34262]: Deploying daemon osd.7 on smithi139 2024-09-05T01:57:25.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:25 smithi103 ceph-mon[30902]: Deploying daemon osd.7 on smithi139 2024-09-05T01:57:26.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:26 smithi139 ceph-mon[34262]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:26.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:26 smithi167 ceph-mon[35122]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:26.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:26 smithi103 ceph-mon[30902]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:28.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:28 smithi167 ceph-mon[35122]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:28.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:28 smithi139 ceph-mon[34262]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:28.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:28 smithi103 ceph-mon[30902]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:29.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:29.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:29.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:29.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:29.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:29.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:29.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:29.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:29.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:30.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:30 smithi167 ceph-mon[35122]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:30.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:30 smithi139 ceph-mon[34262]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:30.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:30 smithi103 ceph-mon[30902]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:32.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:32 smithi167 ceph-mon[35122]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:32.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:32 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:32.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:32 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:32.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:32 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:32.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:32 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:32.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:32 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:32.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:32 smithi139 ceph-mon[34262]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:32.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:32 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:32.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:32 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:32.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:32 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:32.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:32 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:32.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:32 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:32.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:32 smithi103 ceph-mon[30902]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:32.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:32 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:32.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:32 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:32.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:32 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:32.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:32 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:32.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:32 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.011+0000 7ff7e37fe640 1 -- 172.21.15.139:0/2019700747 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff7f4062630 con 0x7ff7dc0a2bb0 2024-09-05T01:57:33.012 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 7 on host 'smithi139' 2024-09-05T01:57:33.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff7dc0a2bb0 msgr2=0x7ff7dc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:33.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff7dc0a2bb0 0x7ff7dc0a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4199280 tx=0x7ff7e8005cc0 comp rx=0 tx=0).stop 2024-09-05T01:57:33.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 msgr2=0x7ff7f4197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:33.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 0x7ff7f4197d60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009230 tx=0x7ff7e4004340 comp rx=0 tx=0).stop 2024-09-05T01:57:33.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 shutdown_connections 2024-09-05T01:57:33.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff7dc0a2bb0 0x7ff7dc0a5070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:33.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f41037c0 0x7ff7f419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:33.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7f4102e00 0x7ff7f41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:33.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 --2- 172.21.15.139:0/2019700747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f4101c00 0x7ff7f4197d60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:33.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 >> 172.21.15.139:0/2019700747 conn(0x7ff7f40fd390 msgr2=0x7ff7f41061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:57:33.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 shutdown_connections 2024-09-05T01:57:33.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:57:33.013+0000 7ff7f8e74640 1 -- 172.21.15.139:0/2019700747 wait complete. 2024-09-05T01:57:33.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:33.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:33.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:33.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:33.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:33.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:33.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:33.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:33.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:33.685 DEBUG:teuthology.orchestra.run.smithi139:osd.7> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.7.service 2024-09-05T01:57:33.688 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2024-09-05T01:57:33.688 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T01:57:33.942 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:57:34.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:34 smithi167 ceph-mon[35122]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:34.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:34 smithi139 ceph-mon[34262]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:34.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:34 smithi103 ceph-mon[30902]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:35.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:35 smithi139 ceph-mon[34262]: from='osd.7 [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:57:35.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:35 smithi139 ceph-mon[34262]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:57:35.185 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 01:57:35 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T01:57:35.101+0000 7f052bdd3640 -1 osd.7 0 log_to_monitors true 2024-09-05T01:57:35.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:35 smithi167 ceph-mon[35122]: from='osd.7 [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:57:35.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:35 smithi167 ceph-mon[35122]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:57:35.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:35 smithi103 ceph-mon[30902]: from='osd.7 [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:57:35.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:35 smithi103 ceph-mon[30902]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:57:36.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:36 smithi139 ceph-mon[34262]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:36.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:36 smithi139 ceph-mon[34262]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T01:57:36.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:36 smithi139 ceph-mon[34262]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T01:57:36.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:36 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:36.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:36 smithi139 ceph-mon[34262]: from='osd.7 [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:57:36.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:36 smithi139 ceph-mon[34262]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:57:36.280 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 01:57:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T01:57:36.155+0000 7f0527d56640 -1 osd.7 0 waiting for initial osdmap 2024-09-05T01:57:36.281 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 01:57:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T01:57:36.162+0000 7f0523b7e640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:57:36.618 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:36 smithi167 ceph-mon[35122]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:36.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:36 smithi167 ceph-mon[35122]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T01:57:36.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:36 smithi167 ceph-mon[35122]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T01:57:36.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:36 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:36.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:36 smithi167 ceph-mon[35122]: from='osd.7 [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:57:36.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:36 smithi167 ceph-mon[35122]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:57:36.621 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:57:36.648 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-09-05T01:57:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:36 smithi103 ceph-mon[30902]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:36 smithi103 ceph-mon[30902]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T01:57:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:36 smithi103 ceph-mon[30902]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T01:57:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:36 smithi103 ceph-mon[30902]: from='osd.7 [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:57:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:36 smithi103 ceph-mon[30902]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:57:36.899 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:57:37.152 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:57:37.152 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T01:57:37.152 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:37.152 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:37.152 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.152 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.153 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:37.153 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:37.153 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:37.153 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.153 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:37.153 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:37.153 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.280 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:37.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:37.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.362+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/1116221361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a14101d10 msgr2=0x7f4a14102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:37.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.362+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/1116221361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a14101d10 0x7f4a14102190 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009900 tx=0x7f4a0002a410 comp rx=0 tx=0).stop 2024-09-05T01:57:37.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.362+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/1116221361 shutdown_connections 2024-09-05T01:57:37.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.362+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/1116221361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a141026d0 0x7f4a14108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:37.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.362+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/1116221361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a14101d10 0x7f4a14102190 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:37.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.362+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/1116221361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14100b10 0x7f4a14100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:37.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.362+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/1116221361 >> 172.21.15.167:0/1116221361 conn(0x7f4a140fc2a0 msgr2=0x7f4a140fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.362+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/1116221361 shutdown_connections 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.362+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/1116221361 wait complete. 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a1b0d4640 1 Processor -- start 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a1b0d4640 1 -- start start 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a1b0d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14100b10 0x7f4a14197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a1b0d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a14101d10 0x7f4a141983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a1b0d4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a141026d0 0x7f4a1419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a1b0d4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1410c620 con 0x7f4a14100b10 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a1b0d4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1410c4a0 con 0x7f4a14101d10 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a1b0d4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1410c7a0 con 0x7f4a141026d0 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a19671640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a141026d0 0x7f4a1419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:57:37.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a19671640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a141026d0 0x7f4a1419f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46492/0 (socket says 172.21.15.167:46492) 2024-09-05T01:57:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a19671640 1 -- 172.21.15.167:0/684908991 learned_addr learned my addr 172.21.15.167:0/684908991 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:57:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a18e70640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14100b10 0x7f4a14197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:57:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a13fff640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a14101d10 0x7f4a141983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:57:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a19671640 1 -- 172.21.15.167:0/684908991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a14101d10 msgr2=0x7f4a141983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a19671640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a14101d10 0x7f4a141983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a19671640 1 -- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14100b10 msgr2=0x7f4a14197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a19671640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14100b10 0x7f4a14197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a19671640 1 -- 172.21.15.167:0/684908991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1419fa90 con 0x7f4a141026d0 2024-09-05T01:57:37.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.363+0000 7f4a19671640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a141026d0 0x7f4a1419f420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00eb10 tx=0x7f4a0c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:57:37.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.364+0000 7f4a11ffb640 1 -- 172.21.15.167:0/684908991 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004b20 con 0x7f4a141026d0 2024-09-05T01:57:37.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.364+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1419fd80 con 0x7f4a141026d0 2024-09-05T01:57:37.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.364+0000 7f4a11ffb640 1 -- 172.21.15.167:0/684908991 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a0c004cc0 con 0x7f4a141026d0 2024-09-05T01:57:37.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.364+0000 7f4a11ffb640 1 -- 172.21.15.167:0/684908991 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c010690 con 0x7f4a141026d0 2024-09-05T01:57:37.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.364+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a141a01d0 con 0x7f4a141026d0 2024-09-05T01:57:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.365+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a141a0460 con 0x7f4a141026d0 2024-09-05T01:57:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.366+0000 7f4a11ffb640 1 -- 172.21.15.167:0/684908991 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4a0c005020 con 0x7f4a141026d0 2024-09-05T01:57:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.366+0000 7f4a11ffb640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4a040a2bb0 0x7f4a040a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:57:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.367+0000 7f4a11ffb640 1 -- 172.21.15.167:0/684908991 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5430+0+0 (secure 0 0 0) 0x7f4a0c0cea60 con 0x7f4a141026d0 2024-09-05T01:57:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.367+0000 7f4a18e70640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4a040a2bb0 0x7f4a040a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:57:37.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.367+0000 7f4a18e70640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4a040a2bb0 0x7f4a040a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4a14101b70 tx=0x7f4a0800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:57:37.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.371+0000 7f4a11ffb640 1 -- 172.21.15.167:0/684908991 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a0c09c500 con 0x7f4a141026d0 2024-09-05T01:57:37.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:37.545+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4a141a0630 con 0x7f4a040a2bb0 2024-09-05T01:57:37.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:37.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi139 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi139 to 5696M 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: osd.7 [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] boot 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: osdmap e48: 8 total, 8 up, 8 in 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:57:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:57:38.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:38 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:57:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:57:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi139 2024-09-05T01:57:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi139 to 5696M 2024-09-05T01:57:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: osd.7 [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] boot 2024-09-05T01:57:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: osdmap e48: 8 total, 8 up, 8 in 2024-09-05T01:57:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:57:38.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:57:38.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:38 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:57:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:57:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi139 2024-09-05T01:57:38.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi139 to 5696M 2024-09-05T01:57:38.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:57:38.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: osd.7 [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] boot 2024-09-05T01:57:38.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: osdmap e48: 8 total, 8 up, 8 in 2024-09-05T01:57:38.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:57:38.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:57:38.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:57:38.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:38 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:39.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:39 smithi167 ceph-mon[35122]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:57:39.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:39 smithi167 ceph-mon[35122]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T01:57:39.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:39 smithi139 ceph-mon[34262]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:57:39.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:39 smithi139 ceph-mon[34262]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T01:57:39.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:39 smithi103 ceph-mon[30902]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:57:39.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:39 smithi103 ceph-mon[30902]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T01:57:40.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:40 smithi167 ceph-mon[35122]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:40.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:40 smithi167 ceph-mon[35122]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T01:57:40.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:40 smithi167 ceph-mon[35122]: osdmap e51: 8 total, 8 up, 8 in 2024-09-05T01:57:40.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:40 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/947481959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b69841-792a-4d17-8406-278c8d11a20a"}]: dispatch 2024-09-05T01:57:40.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:40 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b69841-792a-4d17-8406-278c8d11a20a"}]: dispatch 2024-09-05T01:57:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:40 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5b69841-792a-4d17-8406-278c8d11a20a"}]': finished 2024-09-05T01:57:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:40 smithi167 ceph-mon[35122]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T01:57:40.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:57:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:40 smithi139 ceph-mon[34262]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:40 smithi139 ceph-mon[34262]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T01:57:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:40 smithi139 ceph-mon[34262]: osdmap e51: 8 total, 8 up, 8 in 2024-09-05T01:57:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:40 smithi139 ceph-mon[34262]: from='client.? 172.21.15.167:0/947481959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b69841-792a-4d17-8406-278c8d11a20a"}]: dispatch 2024-09-05T01:57:40.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:40 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b69841-792a-4d17-8406-278c8d11a20a"}]: dispatch 2024-09-05T01:57:40.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:40 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5b69841-792a-4d17-8406-278c8d11a20a"}]': finished 2024-09-05T01:57:40.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:40 smithi139 ceph-mon[34262]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T01:57:40.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:57:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:40 smithi103 ceph-mon[30902]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:40 smithi103 ceph-mon[30902]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T01:57:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:40 smithi103 ceph-mon[30902]: osdmap e51: 8 total, 8 up, 8 in 2024-09-05T01:57:40.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:40 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/947481959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b69841-792a-4d17-8406-278c8d11a20a"}]: dispatch 2024-09-05T01:57:40.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:40 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b69841-792a-4d17-8406-278c8d11a20a"}]: dispatch 2024-09-05T01:57:40.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:40 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5b69841-792a-4d17-8406-278c8d11a20a"}]': finished 2024-09-05T01:57:40.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:40 smithi103 ceph-mon[30902]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T01:57:40.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:57:41.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:41 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/3920400557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:57:41.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:41 smithi139 ceph-mon[34262]: from='client.? 172.21.15.167:0/3920400557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:57:41.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:41 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/3920400557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:57:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:42 smithi167 ceph-mon[35122]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:42 smithi139 ceph-mon[34262]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:42 smithi103 ceph-mon[30902]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:44.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:44 smithi167 ceph-mon[35122]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:44.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:44 smithi139 ceph-mon[34262]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:44.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:44 smithi103 ceph-mon[30902]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:46.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:46 smithi167 ceph-mon[35122]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:46 smithi139 ceph-mon[34262]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:46 smithi103 ceph-mon[30902]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:48.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:48 smithi167 ceph-mon[35122]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:48.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:48 smithi139 ceph-mon[34262]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:48.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:48 smithi103 ceph-mon[30902]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:49.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:49 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T01:57:49.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:49 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:49.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:49 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T01:57:49.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:49 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:49.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:49 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T01:57:49.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:49 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:50.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:50 smithi167 ceph-mon[35122]: Deploying daemon osd.8 on smithi167 2024-09-05T01:57:50.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:50 smithi167 ceph-mon[35122]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:50.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:50 smithi139 ceph-mon[34262]: Deploying daemon osd.8 on smithi167 2024-09-05T01:57:50.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:50 smithi139 ceph-mon[34262]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:50 smithi103 ceph-mon[30902]: Deploying daemon osd.8 on smithi167 2024-09-05T01:57:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:50 smithi103 ceph-mon[30902]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:52.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:52 smithi167 ceph-mon[35122]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:52.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:52 smithi139 ceph-mon[34262]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:52.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:52 smithi103 ceph-mon[30902]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:54 smithi167 ceph-mon[35122]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:54.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:54 smithi139 ceph-mon[34262]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:54.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:54.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:54.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:54 smithi103 ceph-mon[30902]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:54.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:56.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:56 smithi167 ceph-mon[35122]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:56 smithi139 ceph-mon[34262]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:56 smithi103 ceph-mon[30902]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:57.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.752+0000 7f4a11ffb640 1 -- 172.21.15.167:0/684908991 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a141a0630 con 0x7f4a040a2bb0 2024-09-05T01:57:57.753 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2024-09-05T01:57:57.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4a040a2bb0 msgr2=0x7f4a040a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:57.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4a040a2bb0 0x7f4a040a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4a14101b70 tx=0x7f4a0800b390 comp rx=0 tx=0).stop 2024-09-05T01:57:57.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a141026d0 msgr2=0x7f4a1419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:57:57.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a141026d0 0x7f4a1419f420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00eb10 tx=0x7f4a0c00c2d0 comp rx=0 tx=0).stop 2024-09-05T01:57:57.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 shutdown_connections 2024-09-05T01:57:57.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4a040a2bb0 0x7f4a040a5070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:57.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a141026d0 0x7f4a1419f420 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:57.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a14101d10 0x7f4a141983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:57.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 --2- 172.21.15.167:0/684908991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14100b10 0x7f4a14197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:57:57.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 >> 172.21.15.167:0/684908991 conn(0x7f4a140fc2a0 msgr2=0x7f4a1406a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:57:57.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 shutdown_connections 2024-09-05T01:57:57.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:57:57.754+0000 7f4a1b0d4640 1 -- 172.21.15.167:0/684908991 wait complete. 2024-09-05T01:57:58.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:58.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:58.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:57 smithi167 ceph-mon[35122]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:58.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:58.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:57:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:58.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:58.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:57 smithi139 ceph-mon[34262]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:58.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:58.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:57:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:57:58.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:57:58.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:57 smithi103 ceph-mon[30902]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:57:58.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:57:58.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:57:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:57:58.415 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.8.service 2024-09-05T01:57:58.417 INFO:tasks.cephadm:Deploying osd.9 on smithi103 with /dev/vg_nvme/lv_1... 2024-09-05T01:57:58.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T01:57:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:57:59.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 01:57:59 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T01:57:59.734+0000 7f4a679b5640 -1 osd.8 0 log_to_monitors true 2024-09-05T01:58:00.237 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:00 smithi103 ceph-mon[30902]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:58:00.237 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:00 smithi103 ceph-mon[30902]: from='osd.8 [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:58:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:00 smithi103 ceph-mon[30902]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:58:00.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:00 smithi167 ceph-mon[35122]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:58:00.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:00 smithi167 ceph-mon[35122]: from='osd.8 [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:58:00.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:00 smithi167 ceph-mon[35122]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:58:00.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:00 smithi139 ceph-mon[34262]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:58:00.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:00 smithi139 ceph-mon[34262]: from='osd.8 [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:58:00.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:00 smithi139 ceph-mon[34262]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:58:01.292 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 01:58:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T01:58:01.126+0000 7f4a63938640 -1 osd.8 0 waiting for initial osdmap 2024-09-05T01:58:01.293 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 01:58:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T01:58:01.134+0000 7f4a5f760640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:58:01.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:01 smithi167 ceph-mon[35122]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T01:58:01.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:01 smithi167 ceph-mon[35122]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T01:58:01.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:01 smithi167 ceph-mon[35122]: from='osd.8 [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:58:01.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:01 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:01.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:01 smithi167 ceph-mon[35122]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:58:01.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:01 smithi139 ceph-mon[34262]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T01:58:01.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:01 smithi139 ceph-mon[34262]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T01:58:01.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:01 smithi139 ceph-mon[34262]: from='osd.8 [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:58:01.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:01 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:01.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:01 smithi139 ceph-mon[34262]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:58:01.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:01 smithi103 ceph-mon[30902]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T01:58:01.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:01 smithi103 ceph-mon[30902]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T01:58:01.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:01 smithi103 ceph-mon[30902]: from='osd.8 [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:58:01.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:01 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:01.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:01 smithi103 ceph-mon[30902]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:58:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:58:01.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-09-05T01:58:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:58:02.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.172+0000 7ff427e45640 1 -- 172.21.15.103:0/902321416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff420104f90 msgr2=0x7ff420107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:02.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.172+0000 7ff427e45640 1 --2- 172.21.15.103:0/902321416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff420104f90 0x7ff420107380 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff410009900 tx=0x7ff41002a410 comp rx=0 tx=0).stop 2024-09-05T01:58:02.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 -- 172.21.15.103:0/902321416 shutdown_connections 2024-09-05T01:58:02.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 --2- 172.21.15.103:0/902321416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4201078c0 0x7ff420109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:02.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 --2- 172.21.15.103:0/902321416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff420104f90 0x7ff420107380 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:02.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 --2- 172.21.15.103:0/902321416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff420102660 0x7ff420104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:02.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 -- 172.21.15.103:0/902321416 >> 172.21.15.103:0/902321416 conn(0x7ff4200fc280 msgr2=0x7ff4200fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:58:02.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 -- 172.21.15.103:0/902321416 shutdown_connections 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 -- 172.21.15.103:0/902321416 wait complete. 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 Processor -- start 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 -- start start 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff420102660 0x7ff420197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff420104f90 0x7ff4201983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4201078c0 0x7ff42019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff42010d6f0 con 0x7ff4201078c0 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff42010d570 con 0x7ff420102660 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.173+0000 7ff427e45640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff42010d870 con 0x7ff420104f90 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4263e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4201078c0 0x7ff42019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4263e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4201078c0 0x7ff42019f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45154/0 (socket says 172.21.15.103:45154) 2024-09-05T01:58:02.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4263e2640 1 -- 172.21.15.103:0/3707687812 learned_addr learned my addr 172.21.15.103:0/3707687812 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4253e0640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff420104f90 0x7ff4201983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff425be1640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff420102660 0x7ff420197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4263e2640 1 -- 172.21.15.103:0/3707687812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff420104f90 msgr2=0x7ff4201983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4263e2640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff420104f90 0x7ff4201983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4263e2640 1 -- 172.21.15.103:0/3707687812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff420102660 msgr2=0x7ff420197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4263e2640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff420102660 0x7ff420197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4263e2640 1 -- 172.21.15.103:0/3707687812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42019fa90 con 0x7ff4201078c0 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff4263e2640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4201078c0 0x7ff42019f420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff41800eaa0 tx=0x7ff41800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff416ffd640 1 -- 172.21.15.103:0/3707687812 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41800cdb0 con 0x7ff4201078c0 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff416ffd640 1 -- 172.21.15.103:0/3707687812 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff418005020 con 0x7ff4201078c0 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42019fd80 con 0x7ff4201078c0 2024-09-05T01:58:02.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff416ffd640 1 -- 172.21.15.103:0/3707687812 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180116d0 con 0x7ff4201078c0 2024-09-05T01:58:02.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.174+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4201a0230 con 0x7ff4201078c0 2024-09-05T01:58:02.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.175+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4200fdd70 con 0x7ff4201078c0 2024-09-05T01:58:02.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.176+0000 7ff416ffd640 1 -- 172.21.15.103:0/3707687812 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff4180051d0 con 0x7ff4201078c0 2024-09-05T01:58:02.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.177+0000 7ff416ffd640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff40c0a2c30 0x7ff40c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:02.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.177+0000 7ff425be1640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff40c0a2c30 0x7ff40c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:02.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.177+0000 7ff416ffd640 1 -- 172.21.15.103:0/3707687812 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5862+0+0 (secure 0 0 0) 0x7ff4180ce750 con 0x7ff4201078c0 2024-09-05T01:58:02.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.177+0000 7ff425be1640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff40c0a2c30 0x7ff40c0a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00bbe0 tx=0x7ff41c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:58:02.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.181+0000 7ff416ffd640 1 -- 172.21.15.103:0/3707687812 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff418016030 con 0x7ff4201078c0 2024-09-05T01:58:02.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:02.361+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff420062630 con 0x7ff40c0a2c30 2024-09-05T01:58:02.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:58:02.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:02.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:58:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:58:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:58:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:58:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T01:58:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:02.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:02.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:02.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:02.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:02.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:02.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:03.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi167 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi167 to 5696M 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: osd.8 [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] boot 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: osdmap e55: 9 total, 9 up, 9 in 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:58:03.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:03 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:03.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi167 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi167 to 5696M 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: osd.8 [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] boot 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: osdmap e55: 9 total, 9 up, 9 in 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:58:03.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:03 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:03.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:58:03.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:58:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi167 2024-09-05T01:58:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi167 to 5696M 2024-09-05T01:58:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: osd.8 [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] boot 2024-09-05T01:58:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: osdmap e55: 9 total, 9 up, 9 in 2024-09-05T01:58:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:58:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:58:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:58:03.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:03 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:04.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:04 smithi139 ceph-mon[34262]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:58:04.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:04 smithi139 ceph-mon[34262]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:58:04.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:04 smithi139 ceph-mon[34262]: osdmap e56: 9 total, 9 up, 9 in 2024-09-05T01:58:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:04 smithi103 ceph-mon[30902]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:58:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:04 smithi103 ceph-mon[30902]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:58:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:04 smithi103 ceph-mon[30902]: osdmap e56: 9 total, 9 up, 9 in 2024-09-05T01:58:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:04 smithi167 ceph-mon[35122]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:58:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:04 smithi167 ceph-mon[35122]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:58:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:04 smithi167 ceph-mon[35122]: osdmap e56: 9 total, 9 up, 9 in 2024-09-05T01:58:05.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:05 smithi103 ceph-mon[30902]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T01:58:05.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:05 smithi103 ceph-mon[30902]: osdmap e58: 9 total, 9 up, 9 in 2024-09-05T01:58:05.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:05 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3431900692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e609bbd-4a74-489f-8b8b-47bb13175b8d"}]: dispatch 2024-09-05T01:58:05.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:05 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3431900692' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e609bbd-4a74-489f-8b8b-47bb13175b8d"}]': finished 2024-09-05T01:58:05.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:05 smithi103 ceph-mon[30902]: osdmap e59: 10 total, 9 up, 10 in 2024-09-05T01:58:05.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:05 smithi167 ceph-mon[35122]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T01:58:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:05 smithi167 ceph-mon[35122]: osdmap e58: 9 total, 9 up, 9 in 2024-09-05T01:58:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:05 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3431900692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e609bbd-4a74-489f-8b8b-47bb13175b8d"}]: dispatch 2024-09-05T01:58:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:05 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3431900692' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e609bbd-4a74-489f-8b8b-47bb13175b8d"}]': finished 2024-09-05T01:58:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:05 smithi167 ceph-mon[35122]: osdmap e59: 10 total, 9 up, 10 in 2024-09-05T01:58:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:05.542 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:05 smithi139 ceph-mon[34262]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T01:58:05.542 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:05 smithi139 ceph-mon[34262]: osdmap e58: 9 total, 9 up, 9 in 2024-09-05T01:58:05.542 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:05 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3431900692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e609bbd-4a74-489f-8b8b-47bb13175b8d"}]: dispatch 2024-09-05T01:58:05.543 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:05 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3431900692' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e609bbd-4a74-489f-8b8b-47bb13175b8d"}]': finished 2024-09-05T01:58:05.543 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:05 smithi139 ceph-mon[34262]: osdmap e59: 10 total, 9 up, 10 in 2024-09-05T01:58:05.543 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:06.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:06 smithi103 ceph-mon[30902]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:06.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:06 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3550499857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:58:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:06 smithi167 ceph-mon[35122]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:06 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3550499857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:58:06.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:06 smithi139 ceph-mon[34262]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:06.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:06 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3550499857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:58:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:08 smithi167 ceph-mon[35122]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:08.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:08 smithi139 ceph-mon[34262]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:08 smithi103 ceph-mon[30902]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:10.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:10 smithi103 ceph-mon[30902]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-05T01:58:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:10 smithi167 ceph-mon[35122]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-05T01:58:10.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:10 smithi139 ceph-mon[34262]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-05T01:58:12.464 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:12 smithi103 ceph-mon[30902]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T01:58:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:12 smithi167 ceph-mon[35122]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T01:58:12.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:12 smithi139 ceph-mon[34262]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T01:58:14.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:14 smithi103 ceph-mon[30902]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T01:58:14.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T01:58:14.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:14 smithi167 ceph-mon[35122]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T01:58:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T01:58:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:14 smithi139 ceph-mon[34262]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T01:58:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T01:58:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:15.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:15 smithi167 ceph-mon[35122]: Deploying daemon osd.9 on smithi103 2024-09-05T01:58:15.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:15 smithi139 ceph-mon[34262]: Deploying daemon osd.9 on smithi103 2024-09-05T01:58:15.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:15 smithi103 ceph-mon[30902]: Deploying daemon osd.9 on smithi103 2024-09-05T01:58:16.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:16 smithi103 ceph-mon[30902]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-05T01:58:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:16 smithi167 ceph-mon[35122]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-05T01:58:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:16 smithi139 ceph-mon[34262]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-05T01:58:18.567 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:18 smithi103 ceph-mon[30902]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:58:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:18 smithi167 ceph-mon[35122]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:58:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:18 smithi139 ceph-mon[34262]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:58:20.317 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:20 smithi103 ceph-mon[30902]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:58:20.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:20.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:20.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:20 smithi167 ceph-mon[35122]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:58:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:20 smithi139 ceph-mon[34262]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:58:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:22.630 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:22 smithi103 ceph-mon[30902]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:22 smithi167 ceph-mon[35122]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:22 smithi139 ceph-mon[34262]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.466+0000 7ff416ffd640 1 -- 172.21.15.103:0/3707687812 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff420062630 con 0x7ff40c0a2c30 2024-09-05T01:58:23.468 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 9 on host 'smithi103' 2024-09-05T01:58:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.468+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff40c0a2c30 msgr2=0x7ff40c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.468+0000 7ff427e45640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff40c0a2c30 0x7ff40c0a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00bbe0 tx=0x7ff41c00b390 comp rx=0 tx=0).stop 2024-09-05T01:58:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.468+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4201078c0 msgr2=0x7ff42019f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.468+0000 7ff427e45640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4201078c0 0x7ff42019f420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff41800eaa0 tx=0x7ff41800ef60 comp rx=0 tx=0).stop 2024-09-05T01:58:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.469+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 shutdown_connections 2024-09-05T01:58:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.469+0000 7ff427e45640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4201078c0 0x7ff42019f420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.469+0000 7ff427e45640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff40c0a2c30 0x7ff40c0a50f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.469+0000 7ff427e45640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff420104f90 0x7ff4201983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.469+0000 7ff427e45640 1 --2- 172.21.15.103:0/3707687812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff420102660 0x7ff420197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.469+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 >> 172.21.15.103:0/3707687812 conn(0x7ff4200fc280 msgr2=0x7ff420103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:58:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.469+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 shutdown_connections 2024-09-05T01:58:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:58:23.469+0000 7ff427e45640 1 -- 172.21.15.103:0/3707687812 wait complete. 2024-09-05T01:58:23.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:23.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:23.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:23 smithi103 ceph-mon[30902]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:23.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:23.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:23 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:23 smithi167 ceph-mon[35122]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:23 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:23.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:23.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:23 smithi139 ceph-mon[34262]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:23.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:23.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:23.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:23 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:24.168 DEBUG:teuthology.orchestra.run.smithi103:osd.9> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.9.service 2024-09-05T01:58:24.171 INFO:tasks.cephadm:Deploying osd.10 on smithi139 with /dev/vg_nvme/lv_1... 2024-09-05T01:58:24.171 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T01:58:24.427 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:58:25.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 01:58:25 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T01:58:25.590+0000 7fa585c26640 -1 osd.9 0 log_to_monitors true 2024-09-05T01:58:26.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:26 smithi139 ceph-mon[34262]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:26.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:26 smithi139 ceph-mon[34262]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T01:58:26.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:26 smithi167 ceph-mon[35122]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:26.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:26 smithi167 ceph-mon[35122]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T01:58:26.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:26 smithi103 ceph-mon[30902]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:26.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:26 smithi103 ceph-mon[30902]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T01:58:27.268 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-05T01:58:27.292 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_1 2024-09-05T01:58:27.544 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:58:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:27 smithi167 ceph-mon[35122]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T01:58:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:27 smithi167 ceph-mon[35122]: osdmap e60: 10 total, 9 up, 10 in 2024-09-05T01:58:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:27 smithi167 ceph-mon[35122]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:58:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:27 smithi167 ceph-mon[35122]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:58:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:27 smithi167 ceph-mon[35122]: osdmap e61: 10 total, 9 up, 10 in 2024-09-05T01:58:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:27.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:27 smithi139 ceph-mon[34262]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T01:58:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:27 smithi139 ceph-mon[34262]: osdmap e60: 10 total, 9 up, 10 in 2024-09-05T01:58:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:27 smithi139 ceph-mon[34262]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:58:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:27 smithi139 ceph-mon[34262]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:58:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:27 smithi139 ceph-mon[34262]: osdmap e61: 10 total, 9 up, 10 in 2024-09-05T01:58:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:27 smithi103 ceph-mon[30902]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T01:58:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:27 smithi103 ceph-mon[30902]: osdmap e60: 10 total, 9 up, 10 in 2024-09-05T01:58:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:27 smithi103 ceph-mon[30902]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-05T01:58:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:27 smithi103 ceph-mon[30902]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-05T01:58:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:27 smithi103 ceph-mon[30902]: osdmap e61: 10 total, 9 up, 10 in 2024-09-05T01:58:27.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.015+0000 7f4d32c00640 1 -- 172.21.15.139:0/4227475657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 msgr2=0x7f4d2c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.015+0000 7f4d32c00640 1 --2- 172.21.15.139:0/4227475657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 0x7f4d2c102000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009900 tx=0x7f4d2802a410 comp rx=0 tx=0).stop 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.015+0000 7f4d32c00640 1 -- 172.21.15.139:0/4227475657 shutdown_connections 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.015+0000 7f4d32c00640 1 --2- 172.21.15.139:0/4227475657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d2c1037c0 0x7f4d2c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.015+0000 7f4d32c00640 1 --2- 172.21.15.139:0/4227475657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d2c102e00 0x7f4d2c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.015+0000 7f4d32c00640 1 --2- 172.21.15.139:0/4227475657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 0x7f4d2c102000 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.015+0000 7f4d32c00640 1 -- 172.21.15.139:0/4227475657 >> 172.21.15.139:0/4227475657 conn(0x7f4d2c0fd390 msgr2=0x7f4d2c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.015+0000 7f4d32c00640 1 -- 172.21.15.139:0/4227475657 shutdown_connections 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.015+0000 7f4d32c00640 1 -- 172.21.15.139:0/4227475657 wait complete. 2024-09-05T01:58:28.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.016+0000 7f4d32c00640 1 Processor -- start 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.016+0000 7f4d32c00640 1 -- start start 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.016+0000 7f4d32c00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 0x7f4d2c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d32c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d2c102e00 0x7f4d2c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d32c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d2c1037c0 0x7f4d2c19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d32c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10d740 con 0x7f4d2c102e00 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d32c00640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10d5c0 con 0x7f4d2c101c00 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d32c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10d8c0 con 0x7f4d2c1037c0 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3099c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 0x7f4d2c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3099c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 0x7f4d2c197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42590/0 (socket says 172.21.15.139:42590) 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3099c640 1 -- 172.21.15.139:0/1791650334 learned_addr learned my addr 172.21.15.139:0/1791650334 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-05T01:58:28.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3119d640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d2c1037c0 0x7f4d2c19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d23fff640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d2c102e00 0x7f4d2c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3099c640 1 -- 172.21.15.139:0/1791650334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d2c1037c0 msgr2=0x7f4d2c19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3099c640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d2c1037c0 0x7f4d2c19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3099c640 1 -- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d2c102e00 msgr2=0x7f4d2c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3099c640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d2c102e00 0x7f4d2c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3099c640 1 -- 172.21.15.139:0/1791650334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c19fa50 con 0x7f4d2c101c00 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d3099c640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 0x7f4d2c197e20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009230 tx=0x7f4d28004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d21ffb640 1 -- 172.21.15.139:0/1791650334 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2803f030 con 0x7f4d2c101c00 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c19fce0 con 0x7f4d2c101c00 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d21ffb640 1 -- 172.21.15.139:0/1791650334 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d28003cb0 con 0x7f4d2c101c00 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.017+0000 7f4d21ffb640 1 -- 172.21.15.139:0/1791650334 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28035070 con 0x7f4d2c101c00 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.018+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c1a01c0 con 0x7f4d2c101c00 2024-09-05T01:58:28.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.019+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d2c1090a0 con 0x7f4d2c101c00 2024-09-05T01:58:28.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.020+0000 7f4d21ffb640 1 -- 172.21.15.139:0/1791650334 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4d28042030 con 0x7f4d2c101c00 2024-09-05T01:58:28.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.020+0000 7f4d21ffb640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d140a2bb0 0x7f4d140a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:28.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.020+0000 7f4d21ffb640 1 -- 172.21.15.139:0/1791650334 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5981+0+0 (secure 0 0 0) 0x7f4d280eecf0 con 0x7f4d2c101c00 2024-09-05T01:58:28.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.021+0000 7f4d23fff640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d140a2bb0 0x7f4d140a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:28.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.021+0000 7f4d23fff640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d140a2bb0 0x7f4d140a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c199340 tx=0x7f4d1c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:58:28.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.025+0000 7f4d21ffb640 1 -- 172.21.15.139:0/1791650334 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d280f3070 con 0x7f4d2c101c00 2024-09-05T01:58:28.148 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 01:58:27 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T01:58:27.822+0000 7fa581ba9640 -1 osd.9 0 waiting for initial osdmap 2024-09-05T01:58:28.148 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 01:58:27 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T01:58:27.834+0000 7fa5799c9640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:58:28.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:28.199+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4d2c062630 con 0x7f4d140a2bb0 2024-09-05T01:58:28.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:28.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:28.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:28.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:28.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' 2024-09-05T01:58:28.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:28.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:58:28.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:58:28.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:28 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:58:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:58:28.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:28 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651]' entity='osd.9' 2024-09-05T01:58:28.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:28.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:58:28.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:58:28.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:28 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:29.633 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:29 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:58:29.633 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:29 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:58:29.633 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:29 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi103 2024-09-05T01:58:29.633 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:29 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-05T01:58:29.633 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:29 smithi139 ceph-mon[34262]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:58:29.634 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:29 smithi139 ceph-mon[34262]: osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] boot 2024-09-05T01:58:29.634 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:29 smithi139 ceph-mon[34262]: osdmap e62: 10 total, 10 up, 10 in 2024-09-05T01:58:29.634 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:29 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:58:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:29 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:58:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:29 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi103 2024-09-05T01:58:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:29 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-05T01:58:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:29 smithi167 ceph-mon[35122]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:58:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:29 smithi167 ceph-mon[35122]: osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] boot 2024-09-05T01:58:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:29 smithi167 ceph-mon[35122]: osdmap e62: 10 total, 10 up, 10 in 2024-09-05T01:58:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:29.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:29 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:58:29.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:29 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:58:29.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:29 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi103 2024-09-05T01:58:29.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:29 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-05T01:58:29.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:29 smithi103 ceph-mon[30902]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:58:29.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:29 smithi103 ceph-mon[30902]: osd.9 [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] boot 2024-09-05T01:58:29.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:29 smithi103 ceph-mon[30902]: osdmap e62: 10 total, 10 up, 10 in 2024-09-05T01:58:29.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:58:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:30 smithi167 ceph-mon[35122]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:30.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:30 smithi139 ceph-mon[34262]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:30 smithi103 ceph-mon[30902]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:58:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:31 smithi167 ceph-mon[35122]: osdmap e63: 10 total, 10 up, 10 in 2024-09-05T01:58:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:31 smithi167 ceph-mon[35122]: from='client.? 172.21.15.139:0/3550877424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52a68fc4-9a55-4a43-a1da-72019facc4c9"}]: dispatch 2024-09-05T01:58:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:31 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52a68fc4-9a55-4a43-a1da-72019facc4c9"}]: dispatch 2024-09-05T01:58:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:31 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52a68fc4-9a55-4a43-a1da-72019facc4c9"}]': finished 2024-09-05T01:58:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:31 smithi167 ceph-mon[35122]: osdmap e64: 11 total, 10 up, 11 in 2024-09-05T01:58:31.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:31 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:31.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:31 smithi139 ceph-mon[34262]: osdmap e63: 10 total, 10 up, 10 in 2024-09-05T01:58:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:31 smithi139 ceph-mon[34262]: from='client.? 172.21.15.139:0/3550877424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52a68fc4-9a55-4a43-a1da-72019facc4c9"}]: dispatch 2024-09-05T01:58:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:31 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52a68fc4-9a55-4a43-a1da-72019facc4c9"}]: dispatch 2024-09-05T01:58:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:31 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52a68fc4-9a55-4a43-a1da-72019facc4c9"}]': finished 2024-09-05T01:58:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:31 smithi139 ceph-mon[34262]: osdmap e64: 11 total, 10 up, 11 in 2024-09-05T01:58:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:31 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:31.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:31 smithi103 ceph-mon[30902]: osdmap e63: 10 total, 10 up, 10 in 2024-09-05T01:58:31.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:31 smithi103 ceph-mon[30902]: from='client.? 172.21.15.139:0/3550877424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52a68fc4-9a55-4a43-a1da-72019facc4c9"}]: dispatch 2024-09-05T01:58:31.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:31 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52a68fc4-9a55-4a43-a1da-72019facc4c9"}]: dispatch 2024-09-05T01:58:31.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:31 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52a68fc4-9a55-4a43-a1da-72019facc4c9"}]': finished 2024-09-05T01:58:31.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:31 smithi103 ceph-mon[30902]: osdmap e64: 11 total, 10 up, 11 in 2024-09-05T01:58:31.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:31 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:32.549 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:32 smithi103 ceph-mon[30902]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:32.550 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:32 smithi103 ceph-mon[30902]: from='client.? 172.21.15.139:0/1456592807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:58:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:32 smithi167 ceph-mon[35122]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:32 smithi167 ceph-mon[35122]: from='client.? 172.21.15.139:0/1456592807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:58:32.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:32 smithi139 ceph-mon[34262]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:32.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:32 smithi139 ceph-mon[34262]: from='client.? 172.21.15.139:0/1456592807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:58:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:34 smithi167 ceph-mon[35122]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:34.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:34 smithi139 ceph-mon[34262]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:34.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:34 smithi103 ceph-mon[30902]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:36.559 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:36 smithi139 ceph-mon[34262]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:36 smithi167 ceph-mon[35122]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:36 smithi103 ceph-mon[30902]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:38.370 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:38 smithi139 ceph-mon[34262]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:38 smithi167 ceph-mon[35122]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:38 smithi103 ceph-mon[30902]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:40 smithi167 ceph-mon[35122]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T01:58:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:40.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:40 smithi139 ceph-mon[34262]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:40.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T01:58:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:40 smithi103 ceph-mon[30902]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T01:58:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:41.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:41 smithi139 ceph-mon[34262]: Deploying daemon osd.10 on smithi139 2024-09-05T01:58:41.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:41 smithi167 ceph-mon[35122]: Deploying daemon osd.10 on smithi139 2024-09-05T01:58:41.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:41 smithi103 ceph-mon[30902]: Deploying daemon osd.10 on smithi139 2024-09-05T01:58:42.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:42 smithi139 ceph-mon[34262]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:42 smithi167 ceph-mon[35122]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:42 smithi103 ceph-mon[30902]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:44.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:44 smithi139 ceph-mon[34262]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:44 smithi167 ceph-mon[35122]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:44.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:44 smithi103 ceph-mon[30902]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:46 smithi167 ceph-mon[35122]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:46 smithi139 ceph-mon[34262]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:46 smithi103 ceph-mon[30902]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:48.589 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:48 smithi139 ceph-mon[34262]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.628+0000 7f4d21ffb640 1 -- 172.21.15.139:0/1791650334 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4d2c062630 con 0x7f4d140a2bb0 2024-09-05T01:58:48.631 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 10 on host 'smithi139' 2024-09-05T01:58:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d140a2bb0 msgr2=0x7f4d140a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d140a2bb0 0x7f4d140a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c199340 tx=0x7f4d1c00a2d0 comp rx=0 tx=0).stop 2024-09-05T01:58:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 msgr2=0x7f4d2c197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 0x7f4d2c197e20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009230 tx=0x7f4d28004340 comp rx=0 tx=0).stop 2024-09-05T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 shutdown_connections 2024-09-05T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d140a2bb0 0x7f4d140a5070 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d2c1037c0 0x7f4d2c19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d2c102e00 0x7f4d2c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 --2- 172.21.15.139:0/1791650334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101c00 0x7f4d2c197e20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 >> 172.21.15.139:0/1791650334 conn(0x7f4d2c0fd390 msgr2=0x7f4d2c106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 shutdown_connections 2024-09-05T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-05T01:58:48.630+0000 7f4d32c00640 1 -- 172.21.15.139:0/1791650334 wait complete. 2024-09-05T01:58:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:48 smithi167 ceph-mon[35122]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:48.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:48 smithi103 ceph-mon[30902]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:49.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:49 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:49.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:49 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:49.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:49 smithi139 ceph-mon[34262]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:49.795 DEBUG:teuthology.orchestra.run.smithi139:osd.10> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.10.service 2024-09-05T01:58:49.798 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2024-09-05T01:58:49.798 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T01:58:49.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:49 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:49.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:49 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:49.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:49 smithi167 ceph-mon[35122]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:49.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:49 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:49.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:49 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:49.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:49 smithi103 ceph-mon[30902]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:50.059 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:58:51.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:50 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:51.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:50 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:51.379 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:50 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:51.380 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:50 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:51.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:50 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:51.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:50 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:52.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:51 smithi167 ceph-mon[35122]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:52.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:51 smithi167 ceph-mon[35122]: from='osd.10 [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:58:52.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:51 smithi167 ceph-mon[35122]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:58:52.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:51 smithi139 ceph-mon[34262]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:52.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:51 smithi139 ceph-mon[34262]: from='osd.10 [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:58:52.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:51 smithi139 ceph-mon[34262]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:58:52.146 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 01:58:51 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T01:58:51.795+0000 7f209f40e640 -1 osd.10 0 log_to_monitors true 2024-09-05T01:58:52.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:51 smithi103 ceph-mon[30902]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:52.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:51 smithi103 ceph-mon[30902]: from='osd.10 [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:58:52.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:51 smithi103 ceph-mon[30902]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:58:52.962 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-05T01:58:52.989 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-09-05T01:58:53.146 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 01:58:52 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T01:58:52.916+0000 7f209b391640 -1 osd.10 0 waiting for initial osdmap 2024-09-05T01:58:53.146 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 01:58:52 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T01:58:52.925+0000 7f20971b9640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:58:53.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:52 smithi139 ceph-mon[34262]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T01:58:53.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:52 smithi139 ceph-mon[34262]: from='osd.10 [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:58:53.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:52 smithi139 ceph-mon[34262]: osdmap e65: 11 total, 10 up, 11 in 2024-09-05T01:58:53.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:53.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:52 smithi139 ceph-mon[34262]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:58:53.247 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:58:53.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:52 smithi167 ceph-mon[35122]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T01:58:53.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:52 smithi167 ceph-mon[35122]: from='osd.10 [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:58:53.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:52 smithi167 ceph-mon[35122]: osdmap e65: 11 total, 10 up, 11 in 2024-09-05T01:58:53.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:53.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:52 smithi167 ceph-mon[35122]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:58:53.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:52 smithi103 ceph-mon[30902]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T01:58:53.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:52 smithi103 ceph-mon[30902]: from='osd.10 [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:58:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:52 smithi103 ceph-mon[30902]: osdmap e65: 11 total, 10 up, 11 in 2024-09-05T01:58:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:53.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:52 smithi103 ceph-mon[30902]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 -- 172.21.15.167:0/1192872750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818101cd0 msgr2=0x7f2818102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 --2- 172.21.15.167:0/1192872750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818101cd0 0x7f2818102150 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2800009900 tx=0x7f280002a410 comp rx=0 tx=0).stop 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 -- 172.21.15.167:0/1192872750 shutdown_connections 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 --2- 172.21.15.167:0/1192872750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2818102690 0x7f2818108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 --2- 172.21.15.167:0/1192872750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818101cd0 0x7f2818102150 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 --2- 172.21.15.167:0/1192872750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2818100ad0 0x7f2818100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 -- 172.21.15.167:0/1192872750 >> 172.21.15.167:0/1192872750 conn(0x7f28180fc280 msgr2=0x7f28180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 -- 172.21.15.167:0/1192872750 shutdown_connections 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 -- 172.21.15.167:0/1192872750 wait complete. 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.737+0000 7f281d619640 1 Processor -- start 2024-09-05T01:58:53.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f281d619640 1 -- start start 2024-09-05T01:58:53.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f281d619640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2818100ad0 0x7f281806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:53.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f281d619640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2818101cd0 0x7f281819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:53.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f281d619640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818102690 0x7f281819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:53.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f281d619640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f281810c5e0 con 0x7f2818100ad0 2024-09-05T01:58:53.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f281d619640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f281810c460 con 0x7f2818101cd0 2024-09-05T01:58:53.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f281d619640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f281810c760 con 0x7f2818102690 2024-09-05T01:58:53.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818102690 0x7f281819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:53.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818102690 0x7f281819f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47298/0 (socket says 172.21.15.167:47298) 2024-09-05T01:58:53.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28177fe640 1 -- 172.21.15.167:0/145585196 learned_addr learned my addr 172.21.15.167:0/145585196 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28167fc640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2818101cd0 0x7f281819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f2816ffd640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2818100ad0 0x7f281806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28177fe640 1 -- 172.21.15.167:0/145585196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2818101cd0 msgr2=0x7f281819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28177fe640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2818101cd0 0x7f281819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28177fe640 1 -- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2818100ad0 msgr2=0x7f281806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28177fe640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2818100ad0 0x7f281806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28177fe640 1 -- 172.21.15.167:0/145585196 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281819fa00 con 0x7f2818102690 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f28177fe640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818102690 0x7f281819f0f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f281000cce0 tx=0x7f281000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f280ffff640 1 -- 172.21.15.167:0/145585196 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281000f450 con 0x7f2818102690 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f280ffff640 1 -- 172.21.15.167:0/145585196 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f281000faa0 con 0x7f2818102690 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281819fc90 con 0x7f2818102690 2024-09-05T01:58:53.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.738+0000 7f280ffff640 1 -- 172.21.15.167:0/145585196 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810018b70 con 0x7f2818102690 2024-09-05T01:58:53.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.739+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181a0150 con 0x7f2818102690 2024-09-05T01:58:53.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.740+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2818107f70 con 0x7f2818102690 2024-09-05T01:58:53.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.741+0000 7f280ffff640 1 -- 172.21.15.167:0/145585196 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2810018d10 con 0x7f2818102690 2024-09-05T01:58:53.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.741+0000 7f280ffff640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28040a2b80 0x7f28040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:58:53.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.742+0000 7f280ffff640 1 -- 172.21.15.167:0/145585196 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6414+0+0 (secure 0 0 0) 0x7f28100ced60 con 0x7f2818102690 2024-09-05T01:58:53.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.742+0000 7f2816ffd640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28040a2b80 0x7f28040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:58:53.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.742+0000 7f2816ffd640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28040a2b80 0x7f28040a5040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2808005fd0 tx=0x7f2808005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:58:53.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.746+0000 7f280ffff640 1 -- 172.21.15.167:0/145585196 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2810014030 con 0x7f2818102690 2024-09-05T01:58:53.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:58:53.925+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2818062630 con 0x7f28040a2b80 2024-09-05T01:58:54.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:58:54.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: osdmap e66: 11 total, 10 up, 11 in 2024-09-05T01:58:54.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:54.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:54.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi139 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi139 to 4272M 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:54.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:54.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:54.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:53 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:58:54.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: osdmap e66: 11 total, 10 up, 11 in 2024-09-05T01:58:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi139 2024-09-05T01:58:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi139 to 4272M 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:54.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:53 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-05T01:58:54.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: osdmap e66: 11 total, 10 up, 11 in 2024-09-05T01:58:54.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:54.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:54.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:58:54.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi139 2024-09-05T01:58:54.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi139 to 4272M 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:58:54.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:53 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:58:55.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:58:55.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:58:55.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: osd.10 [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] boot 2024-09-05T01:58:55.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: osdmap e67: 11 total, 11 up, 11 in 2024-09-05T01:58:55.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:55.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:58:55.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:58:55.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:58:55.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:55.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:54 smithi167 ceph-mon[35122]: osdmap e68: 11 total, 11 up, 11 in 2024-09-05T01:58:55.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:58:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:58:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: osd.10 [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] boot 2024-09-05T01:58:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: osdmap e67: 11 total, 11 up, 11 in 2024-09-05T01:58:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:58:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:58:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:58:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:54 smithi139 ceph-mon[34262]: osdmap e68: 11 total, 11 up, 11 in 2024-09-05T01:58:55.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:58:55.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:58:55.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: osd.10 [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] boot 2024-09-05T01:58:55.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: osdmap e67: 11 total, 11 up, 11 in 2024-09-05T01:58:55.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:58:55.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:58:55.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:58:55.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:58:55.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:58:55.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:54 smithi103 ceph-mon[30902]: osdmap e68: 11 total, 11 up, 11 in 2024-09-05T01:58:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:55 smithi167 ceph-mon[35122]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:58:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:55 smithi167 ceph-mon[35122]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T01:58:56.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:55 smithi139 ceph-mon[34262]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:58:56.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:55 smithi139 ceph-mon[34262]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T01:58:56.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:55 smithi103 ceph-mon[30902]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:58:56.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:55 smithi103 ceph-mon[30902]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T01:58:57.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:56 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/3080335709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356aa3c1-baca-473e-a6fc-cf366d4783a0"}]: dispatch 2024-09-05T01:58:57.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:56 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356aa3c1-baca-473e-a6fc-cf366d4783a0"}]: dispatch 2024-09-05T01:58:57.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:56 smithi167 ceph-mon[35122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356aa3c1-baca-473e-a6fc-cf366d4783a0"}]': finished 2024-09-05T01:58:57.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:56 smithi167 ceph-mon[35122]: osdmap e70: 12 total, 11 up, 12 in 2024-09-05T01:58:57.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:56 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:58:57.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:56 smithi139 ceph-mon[34262]: from='client.? 172.21.15.167:0/3080335709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356aa3c1-baca-473e-a6fc-cf366d4783a0"}]: dispatch 2024-09-05T01:58:57.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:56 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356aa3c1-baca-473e-a6fc-cf366d4783a0"}]: dispatch 2024-09-05T01:58:57.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:56 smithi139 ceph-mon[34262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356aa3c1-baca-473e-a6fc-cf366d4783a0"}]': finished 2024-09-05T01:58:57.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:56 smithi139 ceph-mon[34262]: osdmap e70: 12 total, 11 up, 12 in 2024-09-05T01:58:57.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:56 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:58:57.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:56 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/3080335709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356aa3c1-baca-473e-a6fc-cf366d4783a0"}]: dispatch 2024-09-05T01:58:57.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:56 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356aa3c1-baca-473e-a6fc-cf366d4783a0"}]: dispatch 2024-09-05T01:58:57.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:56 smithi103 ceph-mon[30902]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356aa3c1-baca-473e-a6fc-cf366d4783a0"}]': finished 2024-09-05T01:58:57.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:56 smithi103 ceph-mon[30902]: osdmap e70: 12 total, 11 up, 12 in 2024-09-05T01:58:57.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:56 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:58:58.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:57 smithi167 ceph-mon[35122]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:58:58.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:57 smithi167 ceph-mon[35122]: from='client.? 172.21.15.167:0/3017424786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:58:58.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:57 smithi167 ceph-mon[35122]: osdmap e71: 12 total, 11 up, 12 in 2024-09-05T01:58:58.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:58:57 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:58:58.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:57 smithi139 ceph-mon[34262]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:58:58.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:57 smithi139 ceph-mon[34262]: from='client.? 172.21.15.167:0/3017424786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:58:58.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:57 smithi139 ceph-mon[34262]: osdmap e71: 12 total, 11 up, 12 in 2024-09-05T01:58:58.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:58:57 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:58:58.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:57 smithi103 ceph-mon[30902]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:58:58.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:57 smithi103 ceph-mon[30902]: from='client.? 172.21.15.167:0/3017424786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:58:58.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:57 smithi103 ceph-mon[30902]: osdmap e71: 12 total, 11 up, 12 in 2024-09-05T01:58:58.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:58:57 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:00.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:00 smithi167 ceph-mon[35122]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-05T01:59:00.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:00 smithi139 ceph-mon[34262]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-05T01:59:00.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:00 smithi103 ceph-mon[30902]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-05T01:59:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:02 smithi167 ceph-mon[35122]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T01:59:02.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:02 smithi139 ceph-mon[34262]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T01:59:02.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:02 smithi103 ceph-mon[30902]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T01:59:04.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:04 smithi167 ceph-mon[35122]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-05T01:59:04.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:04 smithi139 ceph-mon[34262]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-05T01:59:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:04 smithi103 ceph-mon[30902]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-05T01:59:05.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T01:59:05.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:05.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T01:59:05.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:05.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T01:59:05.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:06.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:06 smithi167 ceph-mon[35122]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-05T01:59:06.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:06 smithi167 ceph-mon[35122]: Deploying daemon osd.11 on smithi167 2024-09-05T01:59:06.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:06 smithi139 ceph-mon[34262]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-05T01:59:06.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:06 smithi139 ceph-mon[34262]: Deploying daemon osd.11 on smithi167 2024-09-05T01:59:06.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:06 smithi103 ceph-mon[30902]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-05T01:59:06.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:06 smithi103 ceph-mon[30902]: Deploying daemon osd.11 on smithi167 2024-09-05T01:59:08.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:08 smithi167 ceph-mon[35122]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T01:59:08.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:08 smithi139 ceph-mon[34262]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T01:59:08.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:08 smithi103 ceph-mon[30902]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T01:59:10.379 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:10 smithi167 ceph-mon[35122]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:10.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:10 smithi139 ceph-mon[34262]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:10.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:10 smithi103 ceph-mon[30902]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:11.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:59:11.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:11.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:11.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:59:11.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:11.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:11.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:59:11.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:11.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:12.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:12 smithi103 ceph-mon[30902]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:12.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:12 smithi167 ceph-mon[35122]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:12.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:12 smithi139 ceph-mon[34262]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:14.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:14 smithi167 ceph-mon[35122]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:14.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:14.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:14.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:14.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:59:14.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:14 smithi139 ceph-mon[34262]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:59:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:14 smithi103 ceph-mon[30902]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:14.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:14.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:59:14.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:14.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.722+0000 7f280ffff640 1 -- 172.21.15.167:0/145585196 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2818062630 con 0x7f28040a2b80 2024-09-05T01:59:14.724 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2024-09-05T01:59:14.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28040a2b80 msgr2=0x7f28040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:14.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28040a2b80 0x7f28040a5040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2808005fd0 tx=0x7f2808005850 comp rx=0 tx=0).stop 2024-09-05T01:59:14.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818102690 msgr2=0x7f281819f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:14.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818102690 0x7f281819f0f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f281000cce0 tx=0x7f281000a620 comp rx=0 tx=0).stop 2024-09-05T01:59:14.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 shutdown_connections 2024-09-05T01:59:14.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28040a2b80 0x7f28040a5040 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:14.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818102690 0x7f281819f0f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:14.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2818101cd0 0x7f281819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:14.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 --2- 172.21.15.167:0/145585196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2818100ad0 0x7f281806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:14.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 >> 172.21.15.167:0/145585196 conn(0x7f28180fc280 msgr2=0x7f2818105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:14.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 shutdown_connections 2024-09-05T01:59:14.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:14.725+0000 7f281d619640 1 -- 172.21.15.167:0/145585196 wait complete. 2024-09-05T01:59:15.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:59:15.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:15.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:15.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:59:15.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:15.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:15.468 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.11.service 2024-09-05T01:59:15.470 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-05T01:59:15.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T01:59:15.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:59:15.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:15.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:15.733 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:16.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 -- 172.21.15.103:0/3894744187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102e00 msgr2=0x7f2554103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:16.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 --2- 172.21.15.103:0/3894744187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102e00 0x7f2554103280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f253c009900 tx=0x7f253c02a410 comp rx=0 tx=0).stop 2024-09-05T01:59:16.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 -- 172.21.15.103:0/3894744187 shutdown_connections 2024-09-05T01:59:16.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 --2- 172.21.15.103:0/3894744187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25541037c0 0x7f255410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:16.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 --2- 172.21.15.103:0/3894744187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102e00 0x7f2554103280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 --2- 172.21.15.103:0/3894744187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2554101c00 0x7f2554102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 -- 172.21.15.103:0/3894744187 >> 172.21.15.103:0/3894744187 conn(0x7f25540fd390 msgr2=0x7f25540ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 -- 172.21.15.103:0/3894744187 shutdown_connections 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 -- 172.21.15.103:0/3894744187 wait complete. 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 Processor -- start 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.223+0000 7f255a3b9640 1 -- start start 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f255a3b9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554101c00 0x7f2554197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f255a3b9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2554102e00 0x7f2554198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f255a3b9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25541037c0 0x7f255419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f255a3b9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f255410d710 con 0x7f2554101c00 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f255a3b9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f255410d590 con 0x7f25541037c0 2024-09-05T01:59:16.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f255a3b9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f255410d890 con 0x7f2554102e00 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2553fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554101c00 0x7f2554197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2553fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554101c00 0x7f2554197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47062/0 (socket says 172.21.15.103:47062) 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2553fff640 1 -- 172.21.15.103:0/2664007510 learned_addr learned my addr 172.21.15.103:0/2664007510 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f25537fe640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2554102e00 0x7f2554198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2558956640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25541037c0 0x7f255419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2553fff640 1 -- 172.21.15.103:0/2664007510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2554102e00 msgr2=0x7f2554198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2553fff640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2554102e00 0x7f2554198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2553fff640 1 -- 172.21.15.103:0/2664007510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25541037c0 msgr2=0x7f255419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2553fff640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25541037c0 0x7f255419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2553fff640 1 -- 172.21.15.103:0/2664007510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255419fa00 con 0x7f2554101c00 2024-09-05T01:59:16.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2558956640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25541037c0 0x7f255419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.224+0000 7f2553fff640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554101c00 0x7f2554197dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f254400c970 tx=0x7f254400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.225+0000 7f25517fa640 1 -- 172.21.15.103:0/2664007510 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544004220 con 0x7f2554101c00 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.225+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255419fcf0 con 0x7f2554101c00 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.225+0000 7f25517fa640 1 -- 172.21.15.103:0/2664007510 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25440043c0 con 0x7f2554101c00 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.225+0000 7f25517fa640 1 -- 172.21.15.103:0/2664007510 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254400f450 con 0x7f2554101c00 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.225+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25541a01a0 con 0x7f2554101c00 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.226+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25541090a0 con 0x7f2554101c00 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.228+0000 7f25517fa640 1 -- 172.21.15.103:0/2664007510 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f25440079b0 con 0x7f2554101c00 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.228+0000 7f25517fa640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f25400a2bb0 0x7f25400a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.228+0000 7f25537fe640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f25400a2bb0 0x7f25400a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.228+0000 7f25517fa640 1 -- 172.21.15.103:0/2664007510 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7f25440ceb70 con 0x7f2554101c00 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.229+0000 7f25537fe640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f25400a2bb0 0x7f25400a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f253c000c00 tx=0x7f253c005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:16.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.231+0000 7f25517fa640 1 -- 172.21.15.103:0/2664007510 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2544014030 con 0x7f2554101c00 2024-09-05T01:59:16.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:16 smithi103 ceph-mon[30902]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:16.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.423+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2554062e90 con 0x7f2554101c00 2024-09-05T01:59:16.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.424+0000 7f25517fa640 1 -- 172.21.15.103:0/2664007510 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f254409c0b0 con 0x7f2554101c00 2024-09-05T01:59:16.425 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:16.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f25400a2bb0 msgr2=0x7f25400a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:16.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f25400a2bb0 0x7f25400a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f253c000c00 tx=0x7f253c005e40 comp rx=0 tx=0).stop 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554101c00 msgr2=0x7f2554197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554101c00 0x7f2554197dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f254400c970 tx=0x7f254400ce30 comp rx=0 tx=0).stop 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 shutdown_connections 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25541037c0 0x7f255419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f25400a2bb0 0x7f25400a5070 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2554102e00 0x7f2554198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 --2- 172.21.15.103:0/2664007510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554101c00 0x7f2554197dd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 >> 172.21.15.103:0/2664007510 conn(0x7f25540fd390 msgr2=0x7f2554106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 shutdown_connections 2024-09-05T01:59:16.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:16.426+0000 7f255a3b9640 1 -- 172.21.15.103:0/2664007510 wait complete. 2024-09-05T01:59:16.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:16 smithi167 ceph-mon[35122]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:16 smithi139 ceph-mon[34262]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:17.133 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 01:59:16 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T01:59:16.997+0000 7f3ad6c1d640 -1 osd.11 0 log_to_monitors true 2024-09-05T01:59:17.284 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1725501533,"num_in_osds":12,"osd_in_since":1725501536,"num_remapped_pgs":0} 2024-09-05T01:59:17.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:17 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2664007510' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:59:17.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:17 smithi167 ceph-mon[35122]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T01:59:17.413 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:17 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2664007510' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:59:17.413 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:17 smithi103 ceph-mon[30902]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T01:59:17.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:17 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2664007510' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:59:17.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:17 smithi139 ceph-mon[34262]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T01:59:18.285 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T01:59:18.538 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:18.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:18 smithi167 ceph-mon[35122]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:18.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:18 smithi167 ceph-mon[35122]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T01:59:18.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:18 smithi167 ceph-mon[35122]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T01:59:18.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:18 smithi167 ceph-mon[35122]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:59:18.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:18 smithi139 ceph-mon[34262]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:18 smithi139 ceph-mon[34262]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T01:59:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:18 smithi139 ceph-mon[34262]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T01:59:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:18 smithi139 ceph-mon[34262]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:59:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:18 smithi103 ceph-mon[30902]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:18 smithi103 ceph-mon[30902]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T01:59:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:18 smithi103 ceph-mon[30902]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T01:59:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:18 smithi103 ceph-mon[30902]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-09-05T01:59:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:19.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.023+0000 7f0daf690640 1 -- 172.21.15.103:0/2880365212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da81037c0 msgr2=0x7f0da810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:19.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.023+0000 7f0daf690640 1 --2- 172.21.15.103:0/2880365212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da81037c0 0x7f0da810a050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009ca0 tx=0x7f0da002a8c0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.023+0000 7f0daf690640 1 -- 172.21.15.103:0/2880365212 shutdown_connections 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.023+0000 7f0daf690640 1 --2- 172.21.15.103:0/2880365212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da81037c0 0x7f0da810a050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.023+0000 7f0daf690640 1 --2- 172.21.15.103:0/2880365212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0da8102e00 0x7f0da8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.023+0000 7f0daf690640 1 --2- 172.21.15.103:0/2880365212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da8101c00 0x7f0da8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.023+0000 7f0daf690640 1 -- 172.21.15.103:0/2880365212 >> 172.21.15.103:0/2880365212 conn(0x7f0da80fd390 msgr2=0x7f0da80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.023+0000 7f0daf690640 1 -- 172.21.15.103:0/2880365212 shutdown_connections 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.023+0000 7f0daf690640 1 -- 172.21.15.103:0/2880365212 wait complete. 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.024+0000 7f0daf690640 1 Processor -- start 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.024+0000 7f0daf690640 1 -- start start 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.024+0000 7f0daf690640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da8101c00 0x7f0da8197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:19.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.024+0000 7f0daf690640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da8102e00 0x7f0da8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.024+0000 7f0daf690640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0da81037c0 0x7f0da819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.024+0000 7f0daf690640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0da810d730 con 0x7f0da8102e00 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.024+0000 7f0daf690640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0da810d5b0 con 0x7f0da81037c0 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.024+0000 7f0daf690640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0da810d8b0 con 0x7f0da8101c00 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dacc2b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da8102e00 0x7f0da8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dacc2b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da8102e00 0x7f0da8198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47084/0 (socket says 172.21.15.103:47084) 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dacc2b640 1 -- 172.21.15.103:0/1687512453 learned_addr learned my addr 172.21.15.103:0/1687512453 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dad42c640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da8101c00 0x7f0da8197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dadc2d640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0da81037c0 0x7f0da819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dacc2b640 1 -- 172.21.15.103:0/1687512453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da8101c00 msgr2=0x7f0da8197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dacc2b640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da8101c00 0x7f0da8197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dacc2b640 1 -- 172.21.15.103:0/1687512453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0da81037c0 msgr2=0x7f0da819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dacc2b640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0da81037c0 0x7f0da819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dacc2b640 1 -- 172.21.15.103:0/1687512453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da819fa10 con 0x7f0da8102e00 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0dacc2b640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da8102e00 0x7f0da8198320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800c890 tx=0x7f0d9800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0d9e7fc640 1 -- 172.21.15.103:0/1687512453 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98004220 con 0x7f0da8102e00 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da819fd00 con 0x7f0da8102e00 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0d9e7fc640 1 -- 172.21.15.103:0/1687512453 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d980043c0 con 0x7f0da8102e00 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0d9e7fc640 1 -- 172.21.15.103:0/1687512453 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98004a00 con 0x7f0da8102e00 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.025+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da81a0230 con 0x7f0da8102e00 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.026+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da8109050 con 0x7f0da8102e00 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.027+0000 7f0d9e7fc640 1 -- 172.21.15.103:0/1687512453 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0d98004ba0 con 0x7f0da8102e00 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.028+0000 7f0d9e7fc640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d940a2bb0 0x7f0d940a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.028+0000 7f0dad42c640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d940a2bb0 0x7f0d940a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:19.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.028+0000 7f0d9e7fc640 1 -- 172.21.15.103:0/1687512453 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7f0d980ceb20 con 0x7f0da8102e00 2024-09-05T01:59:19.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.028+0000 7f0dad42c640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d940a2bb0 0x7f0d940a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0da4005fd0 tx=0x7f0da400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:19.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.031+0000 7f0d9e7fc640 1 -- 172.21.15.103:0/1687512453 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d980cee20 con 0x7f0da8102e00 2024-09-05T01:59:19.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 01:59:18 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T01:59:18.836+0000 7f3ad2ba0640 -1 osd.11 0 waiting for initial osdmap 2024-09-05T01:59:19.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 01:59:18 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T01:59:18.851+0000 7f3aca9c0640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:59:19.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.198+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0da802da10 con 0x7f0da8102e00 2024-09-05T01:59:19.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.198+0000 7f0d9e7fc640 1 -- 172.21.15.103:0/1687512453 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f0d98014030 con 0x7f0da8102e00 2024-09-05T01:59:19.200 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:19.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:59:19.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: osdmap e73: 12 total, 11 up, 12 in 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:19.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:59:19.202 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.202 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' 2024-09-05T01:59:19.202 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:19.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.200+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d940a2bb0 msgr2=0x7f0d940a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:19.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d940a2bb0 0x7f0d940a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0da4005fd0 tx=0x7f0da400b390 comp rx=0 tx=0).stop 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da8102e00 msgr2=0x7f0da8198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da8102e00 0x7f0da8198320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800c890 tx=0x7f0d9800cd50 comp rx=0 tx=0).stop 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 shutdown_connections 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0da81037c0 0x7f0da819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0da8102e00 0x7f0da8198320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d940a2bb0 0x7f0d940a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 --2- 172.21.15.103:0/1687512453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da8101c00 0x7f0da8197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 >> 172.21.15.103:0/1687512453 conn(0x7f0da80fd390 msgr2=0x7f0da8106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 shutdown_connections 2024-09-05T01:59:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:19.201+0000 7f0daf690640 1 -- 172.21.15.103:0/1687512453 wait complete. 2024-09-05T01:59:19.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:59:19.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: osdmap e73: 12 total, 11 up, 12 in 2024-09-05T01:59:19.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' 2024-09-05T01:59:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: osdmap e73: 12 total, 11 up, 12 in 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:59:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:19.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:59:19.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:19.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791]' entity='osd.11' 2024-09-05T01:59:19.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:20.026 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1725501533,"num_in_osds":12,"osd_in_since":1725501536,"num_remapped_pgs":0} 2024-09-05T01:59:20.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:20 smithi103 ceph-mon[30902]: purged_snaps scrub starts 2024-09-05T01:59:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:20 smithi103 ceph-mon[30902]: purged_snaps scrub ok 2024-09-05T01:59:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:20 smithi103 ceph-mon[30902]: Detected new or changed devices on smithi167 2024-09-05T01:59:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:20 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi167 to 4272M 2024-09-05T01:59:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:20 smithi103 ceph-mon[30902]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:20 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1687512453' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:59:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:20 smithi103 ceph-mon[30902]: osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] boot 2024-09-05T01:59:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:20 smithi103 ceph-mon[30902]: osdmap e74: 12 total, 12 up, 12 in 2024-09-05T01:59:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:20 smithi167 ceph-mon[35122]: purged_snaps scrub starts 2024-09-05T01:59:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:20 smithi167 ceph-mon[35122]: purged_snaps scrub ok 2024-09-05T01:59:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:20 smithi167 ceph-mon[35122]: Detected new or changed devices on smithi167 2024-09-05T01:59:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:20 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi167 to 4272M 2024-09-05T01:59:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:20 smithi167 ceph-mon[35122]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:20 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1687512453' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:59:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:20 smithi167 ceph-mon[35122]: osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] boot 2024-09-05T01:59:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:20 smithi167 ceph-mon[35122]: osdmap e74: 12 total, 12 up, 12 in 2024-09-05T01:59:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:20 smithi139 ceph-mon[34262]: purged_snaps scrub starts 2024-09-05T01:59:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:20 smithi139 ceph-mon[34262]: purged_snaps scrub ok 2024-09-05T01:59:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:20 smithi139 ceph-mon[34262]: Detected new or changed devices on smithi167 2024-09-05T01:59:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:20 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi167 to 4272M 2024-09-05T01:59:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:20 smithi139 ceph-mon[34262]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:59:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:20 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1687512453' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:59:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:20 smithi139 ceph-mon[34262]: osd.11 [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] boot 2024-09-05T01:59:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:20 smithi139 ceph-mon[34262]: osdmap e74: 12 total, 12 up, 12 in 2024-09-05T01:59:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:59:21.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T01:59:21.287 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.773+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/1250625401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4107980 msgr2=0x7f9cd4109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.773+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/1250625401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4107980 0x7f9cd4109d70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009ca0 tx=0x7f9ccc02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/1250625401 shutdown_connections 2024-09-05T01:59:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/1250625401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4107980 0x7f9cd4109d70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/1250625401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd4105050 0x7f9cd4107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/1250625401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd4102720 0x7f9cd4104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/1250625401 >> 172.21.15.103:0/1250625401 conn(0x7f9cd40fc280 msgr2=0x7f9cd40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:21.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/1250625401 shutdown_connections 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/1250625401 wait complete. 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 Processor -- start 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 -- start start 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4102720 0x7f9cd4197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd4105050 0x7f9cd41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd4107980 0x7f9cd419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd410d7b0 con 0x7f9cd4102720 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd410d630 con 0x7f9cd4105050 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cdc4bc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd410d930 con 0x7f9cd4107980 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cda258640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4102720 0x7f9cd4197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:21.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cda258640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4102720 0x7f9cd4197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37408/0 (socket says 172.21.15.103:37408) 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.774+0000 7f9cda258640 1 -- 172.21.15.103:0/3294810500 learned_addr learned my addr 172.21.15.103:0/3294810500 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cdaa59640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd4107980 0x7f9cd419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cd9a57640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd4105050 0x7f9cd41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cda258640 1 -- 172.21.15.103:0/3294810500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd4107980 msgr2=0x7f9cd419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cda258640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd4107980 0x7f9cd419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cda258640 1 -- 172.21.15.103:0/3294810500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd4105050 msgr2=0x7f9cd41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cda258640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd4105050 0x7f9cd41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cda258640 1 -- 172.21.15.103:0/3294810500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd419fa90 con 0x7f9cd4102720 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cda258640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4102720 0x7f9cd4197e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000d770 tx=0x7f9cd000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:21.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9ccb7fe640 1 -- 172.21.15.103:0/3294810500 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0017070 con 0x7f9cd4102720 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd419fd80 con 0x7f9cd4102720 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9ccb7fe640 1 -- 172.21.15.103:0/3294810500 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd00054d0 con 0x7f9cd4102720 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9ccb7fe640 1 -- 172.21.15.103:0/3294810500 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0004cf0 con 0x7f9cd4102720 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.775+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41a01b0 con 0x7f9cd4102720 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.776+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd40fdd70 con 0x7f9cd4102720 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.777+0000 7f9ccb7fe640 1 -- 172.21.15.103:0/3294810500 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9cd0004e90 con 0x7f9cd4102720 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.778+0000 7f9ccb7fe640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cc00a2bb0 0x7f9cc00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.778+0000 7f9cd9a57640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cc00a2bb0 0x7f9cc00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.778+0000 7f9ccb7fe640 1 -- 172.21.15.103:0/3294810500 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f9cd00ce9f0 con 0x7f9cd4102720 2024-09-05T01:59:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.778+0000 7f9cd9a57640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cc00a2bb0 0x7f9cc00a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4199380 tx=0x7f9cc4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.781+0000 7f9ccb7fe640 1 -- 172.21.15.103:0/3294810500 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9cd009bdf0 con 0x7f9cd4102720 2024-09-05T01:59:21.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.948+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9cd4062e90 con 0x7f9cd4102720 2024-09-05T01:59:21.950 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:21 smithi103 ceph-mon[30902]: osdmap e75: 12 total, 12 up, 12 in 2024-09-05T01:59:21.950 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:21 smithi103 ceph-mon[30902]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:21.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.949+0000 7f9ccb7fe640 1 -- 172.21.15.103:0/3294810500 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f9cd009b6f0 con 0x7f9cd4102720 2024-09-05T01:59:21.951 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:21.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cc00a2bb0 msgr2=0x7f9cc00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cc00a2bb0 0x7f9cc00a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4199380 tx=0x7f9cc4008040 comp rx=0 tx=0).stop 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4102720 msgr2=0x7f9cd4197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4102720 0x7f9cd4197e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000d770 tx=0x7f9cd000dc30 comp rx=0 tx=0).stop 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 shutdown_connections 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cc00a2bb0 0x7f9cc00a5070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd4107980 0x7f9cd419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd4105050 0x7f9cd41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 --2- 172.21.15.103:0/3294810500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd4102720 0x7f9cd4197e60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 >> 172.21.15.103:0/3294810500 conn(0x7f9cd40fc280 msgr2=0x7f9cd41030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 shutdown_connections 2024-09-05T01:59:21.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:21.951+0000 7f9cdc4bc640 1 -- 172.21.15.103:0/3294810500 wait complete. 2024-09-05T01:59:22.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:21 smithi167 ceph-mon[35122]: osdmap e75: 12 total, 12 up, 12 in 2024-09-05T01:59:22.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:21 smithi167 ceph-mon[35122]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:22.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:21 smithi139 ceph-mon[34262]: osdmap e75: 12 total, 12 up, 12 in 2024-09-05T01:59:22.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:21 smithi139 ceph-mon[34262]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:22.749 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1725501559,"num_in_osds":12,"osd_in_since":1725501536,"num_remapped_pgs":0} 2024-09-05T01:59:22.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:59:22.995 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:23.116 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:22 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3294810500' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:59:23.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:22 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3294810500' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:59:23.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:22 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3294810500' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:59:23.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.518+0000 7fb1437f4640 1 -- 172.21.15.103:0/1826974907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102570 msgr2=0x7fb13c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:23.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.518+0000 7fb1437f4640 1 --2- 172.21.15.103:0/1826974907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102570 0x7fb13c1029b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009900 tx=0x7fb12c02a480 comp rx=0 tx=0).stop 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.518+0000 7fb1437f4640 1 -- 172.21.15.103:0/1826974907 shutdown_connections 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.518+0000 7fb1437f4640 1 --2- 172.21.15.103:0/1826974907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c102ef0 0x7fb13c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.518+0000 7fb1437f4640 1 --2- 172.21.15.103:0/1826974907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102570 0x7fb13c1029b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.518+0000 7fb1437f4640 1 --2- 172.21.15.103:0/1826974907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c100e20 0x7fb13c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.518+0000 7fb1437f4640 1 -- 172.21.15.103:0/1826974907 >> 172.21.15.103:0/1826974907 conn(0x7fb13c0fc7d0 msgr2=0x7fb13c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.518+0000 7fb1437f4640 1 -- 172.21.15.103:0/1826974907 shutdown_connections 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.518+0000 7fb1437f4640 1 -- 172.21.15.103:0/1826974907 wait complete. 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb1437f4640 1 Processor -- start 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb1437f4640 1 -- start start 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb1437f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c100e20 0x7fb13c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb1437f4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c102570 0x7fb13c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb1437f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102ef0 0x7fb13c19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb1437f4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c110e70 con 0x7fb13c102ef0 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb1437f4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c110cf0 con 0x7fb13c100e20 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb1437f4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c110ff0 con 0x7fb13c102570 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb141d91640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102ef0 0x7fb13c19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb141d91640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102ef0 0x7fb13c19bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37420/0 (socket says 172.21.15.103:37420) 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb141d91640 1 -- 172.21.15.103:0/4202064184 learned_addr learned my addr 172.21.15.103:0/4202064184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb140d8f640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c102570 0x7fb13c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.519+0000 7fb141590640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c100e20 0x7fb13c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb141d91640 1 -- 172.21.15.103:0/4202064184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c102570 msgr2=0x7fb13c1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb141d91640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c102570 0x7fb13c1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb141d91640 1 -- 172.21.15.103:0/4202064184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c100e20 msgr2=0x7fb13c197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb141d91640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c100e20 0x7fb13c197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb141d91640 1 -- 172.21.15.103:0/4202064184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c19c320 con 0x7fb13c102ef0 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb141d91640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102ef0 0x7fb13c19bc40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb13400ebb0 tx=0x7fb134009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb1327fc640 1 -- 172.21.15.103:0/4202064184 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13400e2e0 con 0x7fb13c102ef0 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb1327fc640 1 -- 172.21.15.103:0/4202064184 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb134007950 con 0x7fb13c102ef0 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c19c5b0 con 0x7fb13c102ef0 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb1327fc640 1 -- 172.21.15.103:0/4202064184 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1340056f0 con 0x7fb13c102ef0 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.520+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c069f70 con 0x7fb13c102ef0 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.521+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13c1021b0 con 0x7fb13c102ef0 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.522+0000 7fb1327fc640 1 -- 172.21.15.103:0/4202064184 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb134005890 con 0x7fb13c102ef0 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.523+0000 7fb1327fc640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb1280a2b80 0x7fb1280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.523+0000 7fb141590640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb1280a2b80 0x7fb1280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.523+0000 7fb1327fc640 1 -- 172.21.15.103:0/4202064184 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb1340d0500 con 0x7fb13c102ef0 2024-09-05T01:59:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.523+0000 7fb141590640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb1280a2b80 0x7fb1280a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb138009990 tx=0x7fb138009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.526+0000 7fb1327fc640 1 -- 172.21.15.103:0/4202064184 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb13409d900 con 0x7fb13c102ef0 2024-09-05T01:59:23.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.693+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb13c198700 con 0x7fb13c102ef0 2024-09-05T01:59:23.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.694+0000 7fb1327fc640 1 -- 172.21.15.103:0/4202064184 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18515 (secure 0 0 0) 0x7fb13c198700 con 0x7fb13c102ef0 2024-09-05T01:59:23.696 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:23.696 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":75,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T01:59:20.844582+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:59:23.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.697+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb1280a2b80 msgr2=0x7fb1280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:23.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.697+0000 7fb1437f4640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb1280a2b80 0x7fb1280a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb138009990 tx=0x7fb138009110 comp rx=0 tx=0).stop 2024-09-05T01:59:23.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.697+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102ef0 msgr2=0x7fb13c19bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:23.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.697+0000 7fb1437f4640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102ef0 0x7fb13c19bc40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb13400ebb0 tx=0x7fb134009f60 comp rx=0 tx=0).stop 2024-09-05T01:59:23.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.698+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 shutdown_connections 2024-09-05T01:59:23.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.698+0000 7fb1437f4640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb13c102ef0 0x7fb13c19bc40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.698+0000 7fb1437f4640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb1280a2b80 0x7fb1280a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.698+0000 7fb1437f4640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb13c102570 0x7fb13c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.698+0000 7fb1437f4640 1 --2- 172.21.15.103:0/4202064184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c100e20 0x7fb13c197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.698+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 >> 172.21.15.103:0/4202064184 conn(0x7fb13c0fc7d0 msgr2=0x7fb13c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.698+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 shutdown_connections 2024-09-05T01:59:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:23.698+0000 7fb1437f4640 1 -- 172.21.15.103:0/4202064184 wait complete. 2024-09-05T01:59:24.095 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:23 smithi103 ceph-mon[30902]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:24.095 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:23 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/4202064184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:59:24.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:23 smithi167 ceph-mon[35122]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:24.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:23 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/4202064184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:59:24.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:23 smithi139 ceph-mon[34262]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:24.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:23 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/4202064184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:59:24.427 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T01:55:35.312049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-05T01:59:24.427 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-05T01:59:24.671 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:25.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.166+0000 7efe3b9ea640 1 -- 172.21.15.103:0/3201658793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 msgr2=0x7efe3410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.166+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/3201658793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 0x7efe3410bb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efe24009900 tx=0x7efe2402a480 comp rx=0 tx=0).stop 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.166+0000 7efe3b9ea640 1 -- 172.21.15.103:0/3201658793 shutdown_connections 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.166+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/3201658793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3410c170 0x7efe3410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.166+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/3201658793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 0x7efe3410bb50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.166+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/3201658793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe34102d40 0x7efe34103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.166+0000 7efe3b9ea640 1 -- 172.21.15.103:0/3201658793 >> 172.21.15.103:0/3201658793 conn(0x7efe340fc7a0 msgr2=0x7efe340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 -- 172.21.15.103:0/3201658793 shutdown_connections 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 -- 172.21.15.103:0/3201658793 wait complete. 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 Processor -- start 2024-09-05T01:59:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 -- start start 2024-09-05T01:59:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe34102d40 0x7efe34198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 0x7efe341987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3410c170 0x7efe3419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe34111e20 con 0x7efe34103660 2024-09-05T01:59:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efe34111ca0 con 0x7efe34102d40 2024-09-05T01:59:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe3b9ea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe34111fa0 con 0x7efe3410c170 2024-09-05T01:59:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.167+0000 7efe38f85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 0x7efe341987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe38f85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 0x7efe341987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37428/0 (socket says 172.21.15.103:37428) 2024-09-05T01:59:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe38f85640 1 -- 172.21.15.103:0/713212714 learned_addr learned my addr 172.21.15.103:0/713212714 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe39f87640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3410c170 0x7efe3419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe39786640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe34102d40 0x7efe34198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe38f85640 1 -- 172.21.15.103:0/713212714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3410c170 msgr2=0x7efe3419cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe38f85640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3410c170 0x7efe3419cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe38f85640 1 -- 172.21.15.103:0/713212714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe34102d40 msgr2=0x7efe34198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe38f85640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe34102d40 0x7efe34198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe38f85640 1 -- 172.21.15.103:0/713212714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3419d230 con 0x7efe34103660 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe38f85640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 0x7efe341987c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efe24006010 tx=0x7efe240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe2a7fc640 1 -- 172.21.15.103:0/713212714 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2403f030 con 0x7efe34103660 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe2a7fc640 1 -- 172.21.15.103:0/713212714 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe24003800 con 0x7efe34103660 2024-09-05T01:59:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe2a7fc640 1 -- 172.21.15.103:0/713212714 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24035070 con 0x7efe34103660 2024-09-05T01:59:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3419d4c0 con 0x7efe34103660 2024-09-05T01:59:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.168+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe341a4e00 con 0x7efe34103660 2024-09-05T01:59:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.169+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3406b070 con 0x7efe34103660 2024-09-05T01:59:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.170+0000 7efe2a7fc640 1 -- 172.21.15.103:0/713212714 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7efe240039b0 con 0x7efe34103660 2024-09-05T01:59:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.171+0000 7efe2a7fc640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efe200a2b80 0x7efe200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.171+0000 7efe39786640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efe200a2b80 0x7efe200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.171+0000 7efe2a7fc640 1 -- 172.21.15.103:0/713212714 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7efe240ef3c0 con 0x7efe34103660 2024-09-05T01:59:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.171+0000 7efe39786640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efe200a2b80 0x7efe200a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe3000abe0 tx=0x7efe3000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.175+0000 7efe2a7fc640 1 -- 172.21.15.103:0/713212714 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe240bc7c0 con 0x7efe34103660 2024-09-05T01:59:25.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.341+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe341071e0 con 0x7efe34103660 2024-09-05T01:59:25.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.341+0000 7efe2a7fc640 1 -- 172.21.15.103:0/713212714 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7efe240bc0c0 con 0x7efe34103660 2024-09-05T01:59:25.343 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-05T01:59:25.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efe200a2b80 msgr2=0x7efe200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:25.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efe200a2b80 0x7efe200a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe3000abe0 tx=0x7efe3000a2d0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 msgr2=0x7efe341987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:25.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 0x7efe341987c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efe24006010 tx=0x7efe240042f0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 shutdown_connections 2024-09-05T01:59:25.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efe200a2b80 0x7efe200a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3410c170 0x7efe3419cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe34103660 0x7efe341987c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 --2- 172.21.15.103:0/713212714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe34102d40 0x7efe34198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:25.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.343+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 >> 172.21.15.103:0/713212714 conn(0x7efe340fc7a0 msgr2=0x7efe340fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:25.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.344+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 shutdown_connections 2024-09-05T01:59:25.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:25.344+0000 7efe3b9ea640 1 -- 172.21.15.103:0/713212714 wait complete. 2024-09-05T01:59:25.992 INFO:tasks.cephadm:Adding mds.a on smithi103 2024-09-05T01:59:25.992 INFO:tasks.cephadm:Adding mds.d on smithi103 2024-09-05T01:59:25.992 INFO:tasks.cephadm:Adding mds.g on smithi103 2024-09-05T01:59:25.992 INFO:tasks.cephadm:Adding mds.j on smithi103 2024-09-05T01:59:25.993 INFO:tasks.cephadm:Adding mds.b on smithi139 2024-09-05T01:59:25.993 INFO:tasks.cephadm:Adding mds.e on smithi139 2024-09-05T01:59:25.993 INFO:tasks.cephadm:Adding mds.h on smithi139 2024-09-05T01:59:25.993 INFO:tasks.cephadm:Adding mds.k on smithi139 2024-09-05T01:59:25.993 INFO:tasks.cephadm:Adding mds.c on smithi167 2024-09-05T01:59:25.993 INFO:tasks.cephadm:Adding mds.f on smithi167 2024-09-05T01:59:25.993 INFO:tasks.cephadm:Adding mds.i on smithi167 2024-09-05T01:59:25.993 INFO:tasks.cephadm:Adding mds.l on smithi167 2024-09-05T01:59:25.993 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph orch apply mds all '12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2024-09-05T01:59:26.253 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:59:26.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:26 smithi167 ceph-mon[35122]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:26.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:26 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/713212714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T01:59:26.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:26 smithi139 ceph-mon[34262]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:26.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:26 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/713212714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T01:59:26.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:26 smithi103 ceph-mon[30902]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:26.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:26 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/713212714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T01:59:26.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.735+0000 7f6f84882640 1 -- 172.21.15.167:0/491207213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 msgr2=0x7f6f80102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:26.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.735+0000 7f6f84882640 1 --2- 172.21.15.167:0/491207213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 0x7f6f80102150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009900 tx=0x7f6f7402a410 comp rx=0 tx=0).stop 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.735+0000 7f6f84882640 1 -- 172.21.15.167:0/491207213 shutdown_connections 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.735+0000 7f6f84882640 1 --2- 172.21.15.167:0/491207213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80102690 0x7f6f80108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.735+0000 7f6f84882640 1 --2- 172.21.15.167:0/491207213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 0x7f6f80102150 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.735+0000 7f6f84882640 1 --2- 172.21.15.167:0/491207213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f80100ad0 0x7f6f80100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.735+0000 7f6f84882640 1 -- 172.21.15.167:0/491207213 >> 172.21.15.167:0/491207213 conn(0x7f6f800fc280 msgr2=0x7f6f800fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.735+0000 7f6f84882640 1 -- 172.21.15.167:0/491207213 shutdown_connections 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.735+0000 7f6f84882640 1 -- 172.21.15.167:0/491207213 wait complete. 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.736+0000 7f6f84882640 1 Processor -- start 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.736+0000 7f6f84882640 1 -- start start 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.736+0000 7f6f84882640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f80100ad0 0x7f6f80104d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.736+0000 7f6f84882640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 0x7f6f80103390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.736+0000 7f6f84882640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80102690 0x7f6f801038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.736+0000 7f6f84882640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010c6b0 con 0x7f6f80100ad0 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.736+0000 7f6f84882640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010c530 con 0x7f6f80102690 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.736+0000 7f6f84882640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010c830 con 0x7f6f80101cd0 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.736+0000 7f6f7dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 0x7f6f80103390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 0x7f6f80103390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41088/0 (socket says 172.21.15.167:41088) 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7dd9b640 1 -- 172.21.15.167:0/3480040480 learned_addr learned my addr 172.21.15.167:0/3480040480 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7e59c640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f80100ad0 0x7f6f80104d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7ed9d640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80102690 0x7f6f801038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7dd9b640 1 -- 172.21.15.167:0/3480040480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80102690 msgr2=0x7f6f801038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7dd9b640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80102690 0x7f6f801038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7dd9b640 1 -- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f80100ad0 msgr2=0x7f6f80104d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7dd9b640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f80100ad0 0x7f6f80104d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7dd9b640 1 -- 172.21.15.167:0/3480040480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f80104190 con 0x7f6f80101cd0 2024-09-05T01:59:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7e59c640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f80100ad0 0x7f6f80104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7ed9d640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80102690 0x7f6f801038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f7dd9b640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 0x7f6f80103390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005940 tx=0x7f6f74004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f6f7fe640 1 -- 172.21.15.167:0/3480040480 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7403f030 con 0x7f6f80101cd0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f6f7fe640 1 -- 172.21.15.167:0/3480040480 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f74003cb0 con 0x7f6f80101cd0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f801a2010 con 0x7f6f80101cd0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f6f7fe640 1 -- 172.21.15.167:0/3480040480 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74035070 con 0x7f6f80101cd0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.737+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f801a2540 con 0x7f6f80101cd0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.738+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f80108b00 con 0x7f6f80101cd0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.740+0000 7f6f6f7fe640 1 -- 172.21.15.167:0/3480040480 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6f74042030 con 0x7f6f80101cd0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.740+0000 7f6f6f7fe640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f680a2b80 0x7f6f680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.741+0000 7f6f6f7fe640 1 -- 172.21.15.167:0/3480040480 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f6f740ef900 con 0x7f6f80101cd0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.741+0000 7f6f7e59c640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f680a2b80 0x7f6f680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.741+0000 7f6f7e59c640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f680a2b80 0x7f6f680a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6f70006fd0 tx=0x7f6f70008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.744+0000 7f6f6f7fe640 1 -- 172.21.15.167:0/3480040480 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f740bcd00 con 0x7f6f80101cd0 2024-09-05T01:59:26.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.930+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) -- 0x7f6f80062730 con 0x7f6f680a2b80 2024-09-05T01:59:26.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.938+0000 7f6f6f7fe640 1 -- 172.21.15.167:0/3480040480 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6f80062730 con 0x7f6f680a2b80 2024-09-05T01:59:26.939 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2024-09-05T01:59:26.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f680a2b80 msgr2=0x7f6f680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f680a2b80 0x7f6f680a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6f70006fd0 tx=0x7f6f70008040 comp rx=0 tx=0).stop 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 msgr2=0x7f6f80103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 0x7f6f80103390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005940 tx=0x7f6f74004430 comp rx=0 tx=0).stop 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 shutdown_connections 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80102690 0x7f6f801038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f680a2b80 0x7f6f680a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f80101cd0 0x7f6f80103390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 --2- 172.21.15.167:0/3480040480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f80100ad0 0x7f6f80104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.940+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 >> 172.21.15.167:0/3480040480 conn(0x7f6f800fc280 msgr2=0x7f6f800fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.941+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 shutdown_connections 2024-09-05T01:59:26.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-05T01:59:26.941+0000 7f6f84882640 1 -- 172.21.15.167:0/3480040480 wait complete. 2024-09-05T01:59:27.670 DEBUG:teuthology.orchestra.run.smithi103:mds.a> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.a.service 2024-09-05T01:59:27.673 DEBUG:teuthology.orchestra.run.smithi103:mds.d> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.d.service 2024-09-05T01:59:27.715 DEBUG:teuthology.orchestra.run.smithi103:mds.g> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.g.service 2024-09-05T01:59:27.758 DEBUG:teuthology.orchestra.run.smithi103:mds.j> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.j.service 2024-09-05T01:59:27.801 DEBUG:teuthology.orchestra.run.smithi139:mds.b> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.b.service 2024-09-05T01:59:27.804 DEBUG:teuthology.orchestra.run.smithi139:mds.e> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.e.service 2024-09-05T01:59:27.847 DEBUG:teuthology.orchestra.run.smithi139:mds.h> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.h.service 2024-09-05T01:59:27.889 DEBUG:teuthology.orchestra.run.smithi139:mds.k> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.k.service 2024-09-05T01:59:27.934 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.c.service 2024-09-05T01:59:27.936 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.f.service 2024-09-05T01:59:27.979 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.i.service 2024-09-05T01:59:28.021 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.l.service 2024-09-05T01:59:28.065 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-05T01:59:28.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:59:28.087 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:59:28.087 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-09-05T01:59:28.088 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:28.088 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:59:28.088 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:28.088 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:28.088 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:59:28.088 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:28.088 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:28.088 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:28.088 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:28.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:28.235 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:59:28.235 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-09-05T01:59:28.235 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:28.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:59:28.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:28.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:28.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:59:28.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:28.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:28.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:28.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:28.307 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 -- 172.21.15.103:0/4253160732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c100e20 msgr2=0x7f067c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 --2- 172.21.15.103:0/4253160732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c100e20 0x7f067c101200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f066c009900 tx=0x7f066c02a480 comp rx=0 tx=0).stop 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 -- 172.21.15.103:0/4253160732 shutdown_connections 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 --2- 172.21.15.103:0/4253160732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c102ef0 0x7f067c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 --2- 172.21.15.103:0/4253160732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f067c102570 0x7f067c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 --2- 172.21.15.103:0/4253160732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c100e20 0x7f067c101200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 -- 172.21.15.103:0/4253160732 >> 172.21.15.103:0/4253160732 conn(0x7f067c0fc7d0 msgr2=0x7f067c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 -- 172.21.15.103:0/4253160732 shutdown_connections 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 -- 172.21.15.103:0/4253160732 wait complete. 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 Processor -- start 2024-09-05T01:59:28.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.765+0000 7f0681c3a640 1 -- start start 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0681c3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f067c100e20 0x7f067c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0681c3a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c102570 0x7f067c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0681c3a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c102ef0 0x7f067c1a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0681c3a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f067c110e30 con 0x7f067c102570 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0681c3a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f067c110cb0 con 0x7f067c100e20 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0681c3a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f067c110fb0 con 0x7f067c102ef0 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0672ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c102570 0x7f067c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0672ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c102570 0x7f067c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37448/0 (socket says 172.21.15.103:37448) 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0672ffd640 1 -- 172.21.15.103:0/344538438 learned_addr learned my addr 172.21.15.103:0/344538438 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f067bfff640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c102ef0 0x7f067c1a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f067b7fe640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f067c100e20 0x7f067c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:28.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0672ffd640 1 -- 172.21.15.103:0/344538438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c102ef0 msgr2=0x7f067c1a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0672ffd640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c102ef0 0x7f067c1a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0672ffd640 1 -- 172.21.15.103:0/344538438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f067c100e20 msgr2=0x7f067c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0672ffd640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f067c100e20 0x7f067c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0672ffd640 1 -- 172.21.15.103:0/344538438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c1a4820 con 0x7f067c102570 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f067b7fe640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f067c100e20 0x7f067c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f067bfff640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c102ef0 0x7f067c1a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0672ffd640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c102570 0x7f067c1997a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f066400d950 tx=0x7f066400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f06797fa640 1 -- 172.21.15.103:0/344538438 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664017070 con 0x7f067c102570 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c1a49f0 con 0x7f067c102570 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f06797fa640 1 -- 172.21.15.103:0/344538438 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0664009c30 con 0x7f067c102570 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f06797fa640 1 -- 172.21.15.103:0/344538438 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066400f620 con 0x7f067c102570 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.766+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c1a4f80 con 0x7f067c102570 2024-09-05T01:59:28.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.767+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c101200 con 0x7f067c102570 2024-09-05T01:59:28.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.769+0000 7f06797fa640 1 -- 172.21.15.103:0/344538438 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f06640109a0 con 0x7f067c102570 2024-09-05T01:59:28.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.769+0000 7f06797fa640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06680a2b80 0x7f06680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:28.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.769+0000 7f067b7fe640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06680a2b80 0x7f06680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:28.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.769+0000 7f06797fa640 1 -- 172.21.15.103:0/344538438 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f06640ce860 con 0x7f067c102570 2024-09-05T01:59:28.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.770+0000 7f067b7fe640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06680a2b80 0x7f06680a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f066c005ab0 tx=0x7f066c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:28.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.772+0000 7f06797fa640 1 -- 172.21.15.103:0/344538438 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0664009de0 con 0x7f067c102570 2024-09-05T01:59:28.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.975+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f067c06a5a0 con 0x7f067c102570 2024-09-05T01:59:28.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.976+0000 7f06797fa640 1 -- 172.21.15.103:0/344538438 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f067c06a5a0 con 0x7f067c102570 2024-09-05T01:59:28.978 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:28.978 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":1,"btime":"2024-09-05T01:52:29:043577+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-05T01:59:28.978 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 1 2024-09-05T01:59:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06680a2b80 msgr2=0x7f06680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06680a2b80 0x7f06680a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f066c005ab0 tx=0x7f066c005a20 comp rx=0 tx=0).stop 2024-09-05T01:59:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c102570 msgr2=0x7f067c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c102570 0x7f067c1997a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f066400d950 tx=0x7f066400de10 comp rx=0 tx=0).stop 2024-09-05T01:59:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 shutdown_connections 2024-09-05T01:59:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06680a2b80 0x7f06680a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c102ef0 0x7f067c1a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f067c102570 0x7f067c1997a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 --2- 172.21.15.103:0/344538438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f067c100e20 0x7f067c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 >> 172.21.15.103:0/344538438 conn(0x7f067c0fc7d0 msgr2=0x7f067c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 shutdown_connections 2024-09-05T01:59:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:28.978+0000 7f0681c3a640 1 -- 172.21.15.103:0/344538438 wait complete. 2024-09-05T01:59:29.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:28 smithi167 ceph-mon[35122]: Deploying daemon mds.l on smithi167 2024-09-05T01:59:29.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:28 smithi139 ceph-mon[34262]: Deploying daemon mds.l on smithi167 2024-09-05T01:59:29.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:28 smithi103 ceph-mon[30902]: Deploying daemon mds.l on smithi167 2024-09-05T01:59:29.587 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 01:59:29 smithi167 systemd[1]: Starting Ceph mds.l for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:29.845 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 01:59:29 smithi167 podman[65410]: 2024-09-05 01:59:29.488498387 +0000 UTC m=+0.015902655 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:59:29.845 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 01:59:29 smithi167 podman[65410]: 2024-09-05 01:59:29.599934373 +0000 UTC m=+0.127338632 container create b511a52653528d21bfa3c0bd19a811094eba3bbbcbd18aea81a0c25b54563177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2) 2024-09-05T01:59:29.845 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 01:59:29 smithi167 podman[65410]: 2024-09-05 01:59:29.844638032 +0000 UTC m=+0.372042293 container init b511a52653528d21bfa3c0bd19a811094eba3bbbcbd18aea81a0c25b54563177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:59:30.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:29 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/344538438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:30.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:29 smithi167 ceph-mon[35122]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:30.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:29 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6832/2095779466,v1:172.21.15.167:6833/2095779466] up:boot 2024-09-05T01:59:30.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:29 smithi167 ceph-mon[35122]: fsmap 1 up:standby 2024-09-05T01:59:30.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:29 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T01:59:30.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 01:59:29 smithi167 podman[65410]: 2024-09-05 01:59:29.847692351 +0000 UTC m=+0.375096605 container start b511a52653528d21bfa3c0bd19a811094eba3bbbcbd18aea81a0c25b54563177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-05T01:59:30.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 01:59:29 smithi167 ceph-mds[65429]: -- 172.21.15.167:0/1763405912 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56284ec33c20 con 0x56284ec11400 2024-09-05T01:59:30.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 01:59:29 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l[65425]: starting mds.l at 2024-09-05T01:59:30.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 01:59:29 smithi167 bash[65410]: b511a52653528d21bfa3c0bd19a811094eba3bbbcbd18aea81a0c25b54563177 2024-09-05T01:59:30.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 01:59:29 smithi167 systemd[1]: Started Ceph mds.l for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:30.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:29 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/344538438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:30.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:29 smithi103 ceph-mon[30902]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:30.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:29 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6832/2095779466,v1:172.21.15.167:6833/2095779466] up:boot 2024-09-05T01:59:30.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:29 smithi103 ceph-mon[30902]: fsmap 1 up:standby 2024-09-05T01:59:30.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:29 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T01:59:30.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:29 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/344538438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:30.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:29 smithi139 ceph-mon[34262]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:30.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:29 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6832/2095779466,v1:172.21.15.167:6833/2095779466] up:boot 2024-09-05T01:59:30.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:29 smithi139 ceph-mon[34262]: fsmap 1 up:standby 2024-09-05T01:59:30.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:29 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T01:59:31.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:30 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:31.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:30 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:31.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:30 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:31.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:30 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:31.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:30 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:31.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:30 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:31.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:30 smithi167 ceph-mon[35122]: Deploying daemon mds.k on smithi139 2024-09-05T01:59:31.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:30 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:31.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:30 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:31.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:30 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:31.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:30 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:31.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:30 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:31.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:30 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:31.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:30 smithi139 ceph-mon[34262]: Deploying daemon mds.k on smithi139 2024-09-05T01:59:31.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:31.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:31.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:31.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:31.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:31.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:30 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:31.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:30 smithi103 ceph-mon[30902]: Deploying daemon mds.k on smithi139 2024-09-05T01:59:31.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:59:32.036 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:32.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:32 smithi139 ceph-mon[34262]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:32.146 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 01:59:32 smithi139 systemd[1]: Starting Ceph mds.k for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:32.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:32 smithi103 ceph-mon[30902]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:32.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:32 smithi167 ceph-mon[35122]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:32.516 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 01:59:32 smithi139 podman[64107]: 2024-09-05 01:59:32.151149719 +0000 UTC m=+0.015700196 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:59:32.516 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 01:59:32 smithi139 podman[64107]: 2024-09-05 01:59:32.27168217 +0000 UTC m=+0.136232691 container create 345bdedd648587d0637ce260c94ce832ae4ade219264e4df9e8cab20b580f379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:59:32.517 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 01:59:32 smithi139 podman[64107]: 2024-09-05 01:59:32.515966293 +0000 UTC m=+0.380516768 container init 345bdedd648587d0637ce260c94ce832ae4ade219264e4df9e8cab20b580f379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:59:32.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.495+0000 7f69f2b67640 1 -- 172.21.15.103:0/751414794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 msgr2=0x7f69ec106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:32.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.495+0000 7f69f2b67640 1 --2- 172.21.15.103:0/751414794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 0x7f69ec106f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009900 tx=0x7f69e002a410 comp rx=0 tx=0).stop 2024-09-05T01:59:32.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.495+0000 7f69f2b67640 1 -- 172.21.15.103:0/751414794 shutdown_connections 2024-09-05T01:59:32.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.495+0000 7f69f2b67640 1 --2- 172.21.15.103:0/751414794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69ec100aa0 0x7f69ec10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.495+0000 7f69f2b67640 1 --2- 172.21.15.103:0/751414794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69ec100100 0x7f69ec100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.495+0000 7f69f2b67640 1 --2- 172.21.15.103:0/751414794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 0x7f69ec106f60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.495+0000 7f69f2b67640 1 -- 172.21.15.103:0/751414794 >> 172.21.15.103:0/751414794 conn(0x7f69ec077230 msgr2=0x7f69ec0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.495+0000 7f69f2b67640 1 -- 172.21.15.103:0/751414794 shutdown_connections 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.495+0000 7f69f2b67640 1 -- 172.21.15.103:0/751414794 wait complete. 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.496+0000 7f69f2b67640 1 Processor -- start 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.496+0000 7f69f2b67640 1 -- start start 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.496+0000 7f69f2b67640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69ec100100 0x7f69ec198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.496+0000 7f69f2b67640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69ec100aa0 0x7f69ec198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f2b67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 0x7f69ec19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f2b67640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec111f40 con 0x7f69ec106b80 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f2b67640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec111dc0 con 0x7f69ec100aa0 2024-09-05T01:59:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f2b67640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec1120c0 con 0x7f69ec100100 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f1104640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 0x7f69ec19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f1104640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 0x7f69ec19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59546/0 (socket says 172.21.15.103:59546) 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f1104640 1 -- 172.21.15.103:0/1646860551 learned_addr learned my addr 172.21.15.103:0/1646860551 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69ebfff640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69ec100aa0 0x7f69ec198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f0903640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69ec100100 0x7f69ec198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f1104640 1 -- 172.21.15.103:0/1646860551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69ec100100 msgr2=0x7f69ec198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f1104640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69ec100100 0x7f69ec198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f1104640 1 -- 172.21.15.103:0/1646860551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69ec100aa0 msgr2=0x7f69ec198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f1104640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69ec100aa0 0x7f69ec198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f1104640 1 -- 172.21.15.103:0/1646860551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ec19d400 con 0x7f69ec106b80 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f0903640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69ec100100 0x7f69ec198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:59:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f1104640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 0x7f69ec19cd20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69e400b9e0 tx=0x7f69e400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69e9ffb640 1 -- 172.21.15.103:0/1646860551 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4019070 con 0x7f69ec106b80 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ec19d6f0 con 0x7f69ec106b80 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69e9ffb640 1 -- 172.21.15.103:0/1646860551 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69e4014440 con 0x7f69ec106b80 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.497+0000 7f69e9ffb640 1 -- 172.21.15.103:0/1646860551 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e40125a0 con 0x7f69ec106b80 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.498+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ec1a4fd0 con 0x7f69ec106b80 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.499+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ec100560 con 0x7f69ec106b80 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.499+0000 7f69e9ffb640 1 -- 172.21.15.103:0/1646860551 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f69e401c070 con 0x7f69ec106b80 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.500+0000 7f69e9ffb640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f69dc0a2b80 0x7f69dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.500+0000 7f69f0903640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f69dc0a2b80 0x7f69dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.500+0000 7f69e9ffb640 1 -- 172.21.15.103:0/1646860551 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f69e40ce0e0 con 0x7f69ec106b80 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.501+0000 7f69f0903640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f69dc0a2b80 0x7f69dc0a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69e0000c00 tx=0x7f69e0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.504+0000 7f69e9ffb640 1 -- 172.21.15.103:0/1646860551 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69e409ae90 con 0x7f69ec106b80 2024-09-05T01:59:32.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.722+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69ec109b30 con 0x7f69ec106b80 2024-09-05T01:59:32.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.722+0000 7f69e9ffb640 1 -- 172.21.15.103:0/1646860551 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f69e409f190 con 0x7f69ec106b80 2024-09-05T01:59:32.725 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:32.725 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":2,"btime":"2024-09-05T01:59:29:875996+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-05T01:59:32.725 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 2 2024-09-05T01:59:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f69dc0a2b80 msgr2=0x7f69dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f69dc0a2b80 0x7f69dc0a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69e0000c00 tx=0x7f69e0031400 comp rx=0 tx=0).stop 2024-09-05T01:59:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 msgr2=0x7f69ec19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 0x7f69ec19cd20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69e400b9e0 tx=0x7f69e400bea0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 shutdown_connections 2024-09-05T01:59:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69ec106b80 0x7f69ec19cd20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69ec100aa0 0x7f69ec198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f69dc0a2b80 0x7f69dc0a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 --2- 172.21.15.103:0/1646860551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69ec100100 0x7f69ec198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:32.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 >> 172.21.15.103:0/1646860551 conn(0x7f69ec077230 msgr2=0x7f69ec105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:32.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 shutdown_connections 2024-09-05T01:59:32.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:32.725+0000 7f69f2b67640 1 -- 172.21.15.103:0/1646860551 wait complete. 2024-09-05T01:59:32.896 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 01:59:32 smithi139 podman[64107]: 2024-09-05 01:59:32.518832215 +0000 UTC m=+0.383382695 container start 345bdedd648587d0637ce260c94ce832ae4ade219264e4df9e8cab20b580f379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-05T01:59:32.896 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 01:59:32 smithi139 ceph-mds[64127]: -- 172.21.15.139:0/3874543628 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c900f7f860 con 0x55c900f5d400 2024-09-05T01:59:32.896 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 01:59:32 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k[64123]: starting mds.k at 2024-09-05T01:59:32.896 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 01:59:32 smithi139 bash[64107]: 345bdedd648587d0637ce260c94ce832ae4ade219264e4df9e8cab20b580f379 2024-09-05T01:59:32.896 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 01:59:32 smithi139 systemd[1]: Started Ceph mds.k for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:33.704 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:33.704 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:33.704 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:33.704 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:33.704 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:33.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:33.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: Deploying daemon mds.j on smithi103 2024-09-05T01:59:33.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1646860551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:33.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6832/4042514104,v1:172.21.15.139:6833/4042514104] up:boot 2024-09-05T01:59:33.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: fsmap 2 up:standby 2024-09-05T01:59:33.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T01:59:33.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:33 smithi103 ceph-mon[30902]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:34.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: Deploying daemon mds.j on smithi103 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1646860551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6832/4042514104,v1:172.21.15.139:6833/4042514104] up:boot 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: fsmap 2 up:standby 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T01:59:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:33 smithi167 ceph-mon[35122]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:34.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: Deploying daemon mds.j on smithi103 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1646860551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6832/4042514104,v1:172.21.15.139:6833/4042514104] up:boot 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: fsmap 2 up:standby 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T01:59:34.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:33 smithi139 ceph-mon[34262]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:35.524 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:59:35.549 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 01:59:35 smithi103 systemd[1]: Starting Ceph mds.j for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:35.846 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 01:59:35 smithi103 podman[67360]: 2024-09-05 01:59:35.480955068 +0000 UTC m=+0.016367046 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:59:35.846 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 01:59:35 smithi103 podman[67360]: 2024-09-05 01:59:35.592568225 +0000 UTC m=+0.127980203 container create b371690aa115daa88771e57552c8b0465b90af9858e31080fa39b228ad38232f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-05T01:59:35.846 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 01:59:35 smithi103 podman[67360]: 2024-09-05 01:59:35.845650935 +0000 UTC m=+0.381062915 container init b371690aa115daa88771e57552c8b0465b90af9858e31080fa39b228ad38232f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-05T01:59:36.013 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:36.132 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 01:59:35 smithi103 podman[67360]: 2024-09-05 01:59:35.84872274 +0000 UTC m=+0.384134728 container start b371690aa115daa88771e57552c8b0465b90af9858e31080fa39b228ad38232f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-05T01:59:36.132 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 01:59:35 smithi103 ceph-mds[67421]: -- 172.21.15.103:0/3685280760 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb5e37d860 con 0x55bb5e35b400 2024-09-05T01:59:36.133 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 01:59:35 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-j[67405]: starting mds.j at 2024-09-05T01:59:36.133 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 01:59:35 smithi103 bash[67360]: b371690aa115daa88771e57552c8b0465b90af9858e31080fa39b228ad38232f 2024-09-05T01:59:36.133 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 01:59:35 smithi103 systemd[1]: Started Ceph mds.j for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:36.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:36 smithi167 ceph-mon[35122]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:36.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:36 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:36.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:36 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:36.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:36 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:36.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:36 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:36.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:36 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:36.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:36 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:36.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:36 smithi139 ceph-mon[34262]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:36.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:36 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:36.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:36 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:36.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:36 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:36.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:36 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:36.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:36 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:36.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:36 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:36.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:36 smithi103 ceph-mon[30902]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:36.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:36.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:36.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:36.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:36.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:36.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:36 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:36.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 -- 172.21.15.103:0/3695357106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 msgr2=0x7f827810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 --2- 172.21.15.103:0/3695357106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 0x7f827810ea90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8270009ca0 tx=0x7f827002a8c0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 -- 172.21.15.103:0/3695357106 shutdown_connections 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 --2- 172.21.15.103:0/3695357106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 0x7f827810ea90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 --2- 172.21.15.103:0/3695357106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8278102790 0x7f82781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 --2- 172.21.15.103:0/3695357106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278101de0 0x7f82781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 -- 172.21.15.103:0/3695357106 >> 172.21.15.103:0/3695357106 conn(0x7f82780fd8c0 msgr2=0x7f82780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 -- 172.21.15.103:0/3695357106 shutdown_connections 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 -- 172.21.15.103:0/3695357106 wait complete. 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 Processor -- start 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.514+0000 7f827e37d640 1 -- start start 2024-09-05T01:59:36.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827e37d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8278101de0 0x7f8278198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827e37d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278102790 0x7f8278198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827e37d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 0x7f827819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827e37d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8278111e20 con 0x7f8278106f10 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827e37d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8278111ca0 con 0x7f8278101de0 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827e37d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8278111fa0 con 0x7f8278102790 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827c91a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 0x7f827819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827c91a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 0x7f827819cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59584/0 (socket says 172.21.15.103:59584) 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827c91a640 1 -- 172.21.15.103:0/940639915 learned_addr learned my addr 172.21.15.103:0/940639915 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f8277fff640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8278101de0 0x7f8278198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827c91a640 1 -- 172.21.15.103:0/940639915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278102790 msgr2=0x7f8278198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:36.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827c91a640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278102790 0x7f8278198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827c91a640 1 -- 172.21.15.103:0/940639915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8278101de0 msgr2=0x7f8278198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827c91a640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8278101de0 0x7f8278198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827c91a640 1 -- 172.21.15.103:0/940639915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827819d1f0 con 0x7f8278106f10 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827c91a640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 0x7f827819cb10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8270009dc0 tx=0x7f8270032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f82757fa640 1 -- 172.21.15.103:0/940639915 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270003cf0 con 0x7f8278106f10 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f82757fa640 1 -- 172.21.15.103:0/940639915 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82700044f0 con 0x7f8278106f10 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f82757fa640 1 -- 172.21.15.103:0/940639915 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270030b80 con 0x7f8278106f10 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827819d480 con 0x7f8278106f10 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.515+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82781a4dc0 con 0x7f8278106f10 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.516+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82781021c0 con 0x7f8278106f10 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.518+0000 7f82757fa640 1 -- 172.21.15.103:0/940639915 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f82700046a0 con 0x7f8278106f10 2024-09-05T01:59:36.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.518+0000 7f82757fa640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82600a2b80 0x7f82600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:36.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.518+0000 7f8277fff640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82600a2b80 0x7f82600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:36.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.518+0000 7f82757fa640 1 -- 172.21.15.103:0/940639915 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f82700f03d0 con 0x7f8278106f10 2024-09-05T01:59:36.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.519+0000 7f8277fff640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82600a2b80 0x7f82600a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f826800abe0 tx=0x7f826800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:36.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.521+0000 7f82757fa640 1 -- 172.21.15.103:0/940639915 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82700bd730 con 0x7f8278106f10 2024-09-05T01:59:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.732+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82781995a0 con 0x7f8278106f10 2024-09-05T01:59:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.733+0000 7f82757fa640 1 -- 172.21.15.103:0/940639915 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f82781995a0 con 0x7f8278106f10 2024-09-05T01:59:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:36.737 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":4,"btime":"2024-09-05T01:59:36:124644+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-05T01:59:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 4 2024-09-05T01:59:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.735+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82600a2b80 msgr2=0x7f82600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.735+0000 7f827e37d640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82600a2b80 0x7f82600a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f826800abe0 tx=0x7f826800a2d0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.735+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 msgr2=0x7f827819cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.735+0000 7f827e37d640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 0x7f827819cb10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8270009dc0 tx=0x7f8270032ff0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.736+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 shutdown_connections 2024-09-05T01:59:36.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.736+0000 7f827e37d640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8278106f10 0x7f827819cb10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.736+0000 7f827e37d640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82600a2b80 0x7f82600a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.736+0000 7f827e37d640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278102790 0x7f8278198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.736+0000 7f827e37d640 1 --2- 172.21.15.103:0/940639915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8278101de0 0x7f8278198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:36.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.736+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 >> 172.21.15.103:0/940639915 conn(0x7f82780fd8c0 msgr2=0x7f82780fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:36.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.736+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 shutdown_connections 2024-09-05T01:59:36.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:36.736+0000 7f827e37d640 1 -- 172.21.15.103:0/940639915 wait complete. 2024-09-05T01:59:37.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:37 smithi139 ceph-mon[34262]: Deploying daemon mds.d on smithi103 2024-09-05T01:59:37.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:37 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.103:6834/726289988,v1:172.21.15.103:6835/726289988] up:boot 2024-09-05T01:59:37.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:37 smithi139 ceph-mon[34262]: fsmap 3 up:standby 2024-09-05T01:59:37.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:37 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T01:59:37.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:37 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/940639915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:37.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:37 smithi103 ceph-mon[30902]: Deploying daemon mds.d on smithi103 2024-09-05T01:59:37.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:37 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.103:6834/726289988,v1:172.21.15.103:6835/726289988] up:boot 2024-09-05T01:59:37.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:37 smithi103 ceph-mon[30902]: fsmap 3 up:standby 2024-09-05T01:59:37.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:37 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T01:59:37.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:37 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/940639915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:37.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:37 smithi167 ceph-mon[35122]: Deploying daemon mds.d on smithi103 2024-09-05T01:59:37.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:37 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.103:6834/726289988,v1:172.21.15.103:6835/726289988] up:boot 2024-09-05T01:59:37.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:37 smithi167 ceph-mon[35122]: fsmap 3 up:standby 2024-09-05T01:59:37.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:37 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T01:59:37.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:37 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/940639915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:38.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:38 smithi139 ceph-mon[34262]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:38.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:38 smithi103 ceph-mon[30902]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:38 smithi167 ceph-mon[35122]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:39.310 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 01:59:39 smithi103 systemd[1]: Starting Ceph mds.d for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:39.582 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 01:59:39 smithi103 podman[67777]: 2024-09-05 01:59:39.21215654 +0000 UTC m=+0.016254439 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:59:39.582 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 01:59:39 smithi103 podman[67777]: 2024-09-05 01:59:39.34623889 +0000 UTC m=+0.150336794 container create eae91d00bc66462f4ced621896da1b32f47f8335f63aaa356eff4ca0a7aa1df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-05T01:59:39.583 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 01:59:39 smithi103 podman[67777]: 2024-09-05 01:59:39.581737392 +0000 UTC m=+0.385835290 container init eae91d00bc66462f4ced621896da1b32f47f8335f63aaa356eff4ca0a7aa1df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-09-05T01:59:39.591 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:59:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:39.898 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 01:59:39 smithi103 podman[67777]: 2024-09-05 01:59:39.584601067 +0000 UTC m=+0.388698970 container start eae91d00bc66462f4ced621896da1b32f47f8335f63aaa356eff4ca0a7aa1df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS) 2024-09-05T01:59:39.899 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 01:59:39 smithi103 ceph-mds[67796]: -- 172.21.15.103:0/3432271784 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d89b45d860 con 0x55d89b43a800 2024-09-05T01:59:39.899 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 01:59:39 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d[67792]: starting mds.d at 2024-09-05T01:59:39.899 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 01:59:39 smithi103 bash[67777]: eae91d00bc66462f4ced621896da1b32f47f8335f63aaa356eff4ca0a7aa1df5 2024-09-05T01:59:39.899 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 01:59:39 smithi103 systemd[1]: Started Ceph mds.d for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:40.270 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:40 smithi103 ceph-mon[30902]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:40.270 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:40.270 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:40.270 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:40.271 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:40.271 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:40.271 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:40 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:40.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.350+0000 7f7a32f02640 1 -- 172.21.15.103:0/3756054556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c100e20 msgr2=0x7f7a2c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.350+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3756054556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c100e20 0x7f7a2c101200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009900 tx=0x7f7a2002a480 comp rx=0 tx=0).stop 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.350+0000 7f7a32f02640 1 -- 172.21.15.103:0/3756054556 shutdown_connections 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.350+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3756054556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a2c102ef0 0x7f7a2c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.350+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3756054556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c102570 0x7f7a2c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.350+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3756054556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c100e20 0x7f7a2c101200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.350+0000 7f7a32f02640 1 -- 172.21.15.103:0/3756054556 >> 172.21.15.103:0/3756054556 conn(0x7f7a2c0fc7d0 msgr2=0x7f7a2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.350+0000 7f7a32f02640 1 -- 172.21.15.103:0/3756054556 shutdown_connections 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.350+0000 7f7a32f02640 1 -- 172.21.15.103:0/3756054556 wait complete. 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a32f02640 1 Processor -- start 2024-09-05T01:59:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a32f02640 1 -- start start 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a32f02640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c100e20 0x7f7a2c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a32f02640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a2c102570 0x7f7a2c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a32f02640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c102ef0 0x7f7a2c1a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a32f02640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c110e50 con 0x7f7a2c102ef0 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a32f02640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c110cd0 con 0x7f7a2c102570 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a32f02640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c110fd0 con 0x7f7a2c100e20 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a3149f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c102ef0 0x7f7a2c1a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a3149f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c102ef0 0x7f7a2c1a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41880/0 (socket says 172.21.15.103:41880) 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.351+0000 7f7a3149f640 1 -- 172.21.15.103:0/3558045246 learned_addr learned my addr 172.21.15.103:0/3558045246 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a2bfff640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a2c102570 0x7f7a2c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a30c9e640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c100e20 0x7f7a2c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:40.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a3149f640 1 -- 172.21.15.103:0/3558045246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c100e20 msgr2=0x7f7a2c199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a3149f640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c100e20 0x7f7a2c199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a3149f640 1 -- 172.21.15.103:0/3558045246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a2c102570 msgr2=0x7f7a2c199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a3149f640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a2c102570 0x7f7a2c199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a3149f640 1 -- 172.21.15.103:0/3558045246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c1a48e0 con 0x7f7a2c102ef0 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a30c9e640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c100e20 0x7f7a2c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a3149f640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c102ef0 0x7f7a2c1a43a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400eaa0 tx=0x7f7a2400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a29ffb640 1 -- 172.21.15.103:0/3558045246 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2400cdb0 con 0x7f7a2c102ef0 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a29ffb640 1 -- 172.21.15.103:0/3558045246 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a24005020 con 0x7f7a2c102ef0 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c1a4ab0 con 0x7f7a2c102ef0 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a29ffb640 1 -- 172.21.15.103:0/3558045246 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a240116d0 con 0x7f7a2c102ef0 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.352+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c1a4f70 con 0x7f7a2c102ef0 2024-09-05T01:59:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.353+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c101200 con 0x7f7a2c102ef0 2024-09-05T01:59:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.354+0000 7f7a29ffb640 1 -- 172.21.15.103:0/3558045246 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7a240051d0 con 0x7f7a2c102ef0 2024-09-05T01:59:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.355+0000 7f7a29ffb640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f7a1c0a2b80 0x7f7a1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.355+0000 7f7a30c9e640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f7a1c0a2b80 0x7f7a1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.355+0000 7f7a29ffb640 1 -- 172.21.15.103:0/3558045246 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f7a240cee50 con 0x7f7a2c102ef0 2024-09-05T01:59:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.356+0000 7f7a30c9e640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f7a1c0a2b80 0x7f7a1c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a20000c00 tx=0x7f7a20032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.359+0000 7f7a29ffb640 1 -- 172.21.15.103:0/3558045246 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a24016030 con 0x7f7a2c102ef0 2024-09-05T01:59:40.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:40 smithi139 ceph-mon[34262]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:40.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:40.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:40.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:40.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:40.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:40.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:40 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:40.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.561+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a2c109090 con 0x7f7a2c102ef0 2024-09-05T01:59:40.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.562+0000 7f7a29ffb640 1 -- 172.21.15.103:0/3558045246 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f7a2409c250 con 0x7f7a2c102ef0 2024-09-05T01:59:40.562 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:40.562 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":5,"btime":"2024-09-05T01:59:40:142180+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-05T01:59:40.563 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 5 2024-09-05T01:59:40.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f7a1c0a2b80 msgr2=0x7f7a1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f7a1c0a2b80 0x7f7a1c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a20000c00 tx=0x7f7a20032c70 comp rx=0 tx=0).stop 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c102ef0 msgr2=0x7f7a2c1a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c102ef0 0x7f7a2c1a43a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400eaa0 tx=0x7f7a2400ef60 comp rx=0 tx=0).stop 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 shutdown_connections 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a2c102ef0 0x7f7a2c1a43a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a2c102570 0x7f7a2c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f7a1c0a2b80 0x7f7a1c0a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 --2- 172.21.15.103:0/3558045246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c100e20 0x7f7a2c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 >> 172.21.15.103:0/3558045246 conn(0x7f7a2c0fc7d0 msgr2=0x7f7a2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 shutdown_connections 2024-09-05T01:59:40.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:40.564+0000 7f7a32f02640 1 -- 172.21.15.103:0/3558045246 wait complete. 2024-09-05T01:59:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:40 smithi167 ceph-mon[35122]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:40.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:40.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:40.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:40 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:41.368 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:41 smithi103 ceph-mon[30902]: Deploying daemon mds.e on smithi139 2024-09-05T01:59:41.369 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:41 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.103:6836/2025990895,v1:172.21.15.103:6837/2025990895] up:boot 2024-09-05T01:59:41.369 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:41 smithi103 ceph-mon[30902]: fsmap 4 up:standby 2024-09-05T01:59:41.369 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:41 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T01:59:41.369 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:41 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3558045246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:41.457 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:41 smithi139 ceph-mon[34262]: Deploying daemon mds.e on smithi139 2024-09-05T01:59:41.457 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:41 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.103:6836/2025990895,v1:172.21.15.103:6837/2025990895] up:boot 2024-09-05T01:59:41.457 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:41 smithi139 ceph-mon[34262]: fsmap 4 up:standby 2024-09-05T01:59:41.457 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:41 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T01:59:41.458 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:41 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3558045246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:41.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:41 smithi167 ceph-mon[35122]: Deploying daemon mds.e on smithi139 2024-09-05T01:59:41.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:41 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.103:6836/2025990895,v1:172.21.15.103:6837/2025990895] up:boot 2024-09-05T01:59:41.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:41 smithi167 ceph-mon[35122]: fsmap 4 up:standby 2024-09-05T01:59:41.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:41 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T01:59:41.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:41 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3558045246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:42.146 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 01:59:41 smithi139 systemd[1]: Starting Ceph mds.e for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:42.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:42 smithi139 ceph-mon[34262]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:42.463 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 01:59:42 smithi139 podman[64421]: 2024-09-05 01:59:42.055232737 +0000 UTC m=+0.015945010 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:59:42.463 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 01:59:42 smithi139 podman[64421]: 2024-09-05 01:59:42.184427332 +0000 UTC m=+0.145139604 container create e04809ef0a728af92579e0ee3550526e49b32873bbefe72bc64d1a03acb748df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-05T01:59:42.463 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 01:59:42 smithi139 podman[64421]: 2024-09-05 01:59:42.462118077 +0000 UTC m=+0.422830352 container init e04809ef0a728af92579e0ee3550526e49b32873bbefe72bc64d1a03acb748df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0) 2024-09-05T01:59:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:42 smithi167 ceph-mon[35122]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:42 smithi103 ceph-mon[30902]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:42.896 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 01:59:42 smithi139 podman[64421]: 2024-09-05 01:59:42.464843078 +0000 UTC m=+0.425555350 container start e04809ef0a728af92579e0ee3550526e49b32873bbefe72bc64d1a03acb748df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-05T01:59:42.896 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 01:59:42 smithi139 ceph-mds[64440]: -- 172.21.15.139:0/3941578027 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f169797860 con 0x55f169775400 2024-09-05T01:59:42.896 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 01:59:42 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e[64436]: starting mds.e at 2024-09-05T01:59:42.896 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 01:59:42 smithi139 bash[64421]: e04809ef0a728af92579e0ee3550526e49b32873bbefe72bc64d1a03acb748df 2024-09-05T01:59:42.896 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 01:59:42 smithi139 systemd[1]: Started Ceph mds.e for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:43.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:59:43.663 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:43.784 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:43.784 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:43.784 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:43.784 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:43.784 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:43.784 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:43.784 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: Deploying daemon mds.f on smithi167 2024-09-05T01:59:43.785 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:43.785 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] up:boot 2024-09-05T01:59:43.785 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: fsmap 5 up:standby 2024-09-05T01:59:43.785 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:43 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T01:59:43.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:43.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:43.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:43.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:43.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:43.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:43.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: Deploying daemon mds.f on smithi167 2024-09-05T01:59:43.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:43.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] up:boot 2024-09-05T01:59:43.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: fsmap 5 up:standby 2024-09-05T01:59:43.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:43 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T01:59:43.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: Deploying daemon mds.f on smithi167 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] up:boot 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: fsmap 5 up:standby 2024-09-05T01:59:43.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:43 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T01:59:44.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 -- 172.21.15.103:0/2700758577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82841027a0 msgr2=0x7f82841067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:44.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 --2- 172.21.15.103:0/2700758577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82841027a0 0x7f82841067f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8274009ca0 tx=0x7f827402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 -- 172.21.15.103:0/2700758577 shutdown_connections 2024-09-05T01:59:44.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 --2- 172.21.15.103:0/2700758577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8284106f20 0x7f828410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 --2- 172.21.15.103:0/2700758577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82841027a0 0x7f82841067f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 --2- 172.21.15.103:0/2700758577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8284101df0 0x7f82841021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 -- 172.21.15.103:0/2700758577 >> 172.21.15.103:0/2700758577 conn(0x7f82840fd8b0 msgr2=0x7f82840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:44.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 -- 172.21.15.103:0/2700758577 shutdown_connections 2024-09-05T01:59:44.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 -- 172.21.15.103:0/2700758577 wait complete. 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 Processor -- start 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.168+0000 7f828ab1a640 1 -- start start 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f828ab1a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8284101df0 0x7f8284198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f828ab1a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82841027a0 0x7f82841988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f828ab1a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8284106f20 0x7f828419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f828ab1a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8284111e30 con 0x7f8284101df0 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f828ab1a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8284111cb0 con 0x7f8284106f20 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f828ab1a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8284111fb0 con 0x7f82841027a0 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82888b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8284101df0 0x7f8284198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82888b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8284101df0 0x7f8284198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41906/0 (socket says 172.21.15.103:41906) 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82888b6640 1 -- 172.21.15.103:0/4123118852 learned_addr learned my addr 172.21.15.103:0/4123118852 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f8283fff640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82841027a0 0x7f82841988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:44.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82890b7640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8284106f20 0x7f828419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82888b6640 1 -- 172.21.15.103:0/4123118852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82841027a0 msgr2=0x7f82841988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82888b6640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82841027a0 0x7f82841988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82888b6640 1 -- 172.21.15.103:0/4123118852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8284106f20 msgr2=0x7f828419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82888b6640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8284106f20 0x7f828419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82888b6640 1 -- 172.21.15.103:0/4123118852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828419d340 con 0x7f8284101df0 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82890b7640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8284106f20 0x7f828419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f82888b6640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8284101df0 0x7f8284198390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c970 tx=0x7f826c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f8281ffb640 1 -- 172.21.15.103:0/4123118852 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c007bb0 con 0x7f8284101df0 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828419d630 con 0x7f8284101df0 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f8281ffb640 1 -- 172.21.15.103:0/4123118852 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f826c007d50 con 0x7f8284101df0 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.169+0000 7f8281ffb640 1 -- 172.21.15.103:0/4123118852 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c00f450 con 0x7f8284101df0 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.170+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82841a4f10 con 0x7f8284101df0 2024-09-05T01:59:44.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.171+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82841021d0 con 0x7f8284101df0 2024-09-05T01:59:44.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.172+0000 7f8281ffb640 1 -- 172.21.15.103:0/4123118852 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f826c004f20 con 0x7f8284101df0 2024-09-05T01:59:44.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.172+0000 7f8281ffb640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82700a2b80 0x7f82700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:44.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.172+0000 7f8283fff640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82700a2b80 0x7f82700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:44.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.172+0000 7f8281ffb640 1 -- 172.21.15.103:0/4123118852 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f826c0ce9e0 con 0x7f8284101df0 2024-09-05T01:59:44.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.173+0000 7f8283fff640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82700a2b80 0x7f82700a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8274009a60 tx=0x7f82740093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:44.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.176+0000 7f8281ffb640 1 -- 172.21.15.103:0/4123118852 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f826c014030 con 0x7f8284101df0 2024-09-05T01:59:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.377+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f828410a190 con 0x7f8284101df0 2024-09-05T01:59:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.378+0000 7f8281ffb640 1 -- 172.21.15.103:0/4123118852 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4747 (secure 0 0 0) 0x7f826c09bde0 con 0x7f8284101df0 2024-09-05T01:59:44.379 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:44.379 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":6,"btime":"2024-09-05T01:59:43:155591+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-05T01:59:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 6 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82700a2b80 msgr2=0x7f82700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82700a2b80 0x7f82700a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8274009a60 tx=0x7f82740093f0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8284101df0 msgr2=0x7f8284198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8284101df0 0x7f8284198390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c970 tx=0x7f826c00ce30 comp rx=0 tx=0).stop 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 shutdown_connections 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8284106f20 0x7f828419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f82700a2b80 0x7f82700a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82841027a0 0x7f82841988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 --2- 172.21.15.103:0/4123118852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8284101df0 0x7f8284198390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 >> 172.21.15.103:0/4123118852 conn(0x7f82840fd8b0 msgr2=0x7f82840fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 shutdown_connections 2024-09-05T01:59:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:44.380+0000 7f828ab1a640 1 -- 172.21.15.103:0/4123118852 wait complete. 2024-09-05T01:59:44.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:44 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/4123118852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:44.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:44 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/4123118852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:44.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:44 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/4123118852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:45.302 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 01:59:45 smithi167 systemd[1]: Starting Ceph mds.f for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:45.592 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 01:59:45 smithi167 podman[65716]: 2024-09-05 01:59:45.203994277 +0000 UTC m=+0.015858480 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:59:45.593 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 01:59:45 smithi167 podman[65716]: 2024-09-05 01:59:45.330765007 +0000 UTC m=+0.142629207 container create 044031dfc5a47254d53a18d6d32dcdbfbe5f7c431910bb53d342b9db35f93a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:59:45.593 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 01:59:45 smithi167 podman[65716]: 2024-09-05 01:59:45.592242485 +0000 UTC m=+0.404106685 container init 044031dfc5a47254d53a18d6d32dcdbfbe5f7c431910bb53d342b9db35f93a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T01:59:45.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:45 smithi139 ceph-mon[34262]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:45.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:45 smithi167 ceph-mon[35122]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:45.886 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 01:59:45 smithi167 podman[65716]: 2024-09-05 01:59:45.595141507 +0000 UTC m=+0.407005709 container start 044031dfc5a47254d53a18d6d32dcdbfbe5f7c431910bb53d342b9db35f93a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:59:45.886 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 01:59:45 smithi167 ceph-mds[65735]: -- 172.21.15.167:0/1112708928 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5e7f65860 con 0x55d5e7f42800 2024-09-05T01:59:45.886 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 01:59:45 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f[65731]: starting mds.f at 2024-09-05T01:59:45.886 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 01:59:45 smithi167 bash[65716]: 044031dfc5a47254d53a18d6d32dcdbfbe5f7c431910bb53d342b9db35f93a20 2024-09-05T01:59:45.886 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 01:59:45 smithi167 systemd[1]: Started Ceph mds.f for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:45.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:45 smithi103 ceph-mon[30902]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:46.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6834/1110459707,v1:172.21.15.167:6835/1110459707] up:boot 2024-09-05T01:59:46.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: fsmap 6 up:standby 2024-09-05T01:59:46.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T01:59:46.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:46.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:46.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:46.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:46.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:46.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:46.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:46 smithi167 ceph-mon[35122]: Deploying daemon mds.c on smithi167 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6834/1110459707,v1:172.21.15.167:6835/1110459707] up:boot 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: fsmap 6 up:standby 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:46.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:46 smithi139 ceph-mon[34262]: Deploying daemon mds.c on smithi167 2024-09-05T01:59:46.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6834/1110459707,v1:172.21.15.167:6835/1110459707] up:boot 2024-09-05T01:59:46.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: fsmap 6 up:standby 2024-09-05T01:59:46.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T01:59:46.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:46.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:46.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:46.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:46.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:46.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:46.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:46 smithi103 ceph-mon[30902]: Deploying daemon mds.c on smithi167 2024-09-05T01:59:47.205 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:59:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:47.884 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:47 smithi103 ceph-mon[30902]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:47.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:47 smithi167 ceph-mon[35122]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:47.886 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 01:59:47 smithi167 systemd[1]: Starting Ceph mds.c for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:48.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.967+0000 7ff5cce39640 1 -- 172.21.15.103:0/4079629852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 msgr2=0x7ff5c8110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:48.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.967+0000 7ff5cce39640 1 --2- 172.21.15.103:0/4079629852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 0x7ff5c8110820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009ca0 tx=0x7ff5c002a950 comp rx=0 tx=0).stop 2024-09-05T01:59:48.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.967+0000 7ff5cce39640 1 -- 172.21.15.103:0/4079629852 shutdown_connections 2024-09-05T01:59:48.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.967+0000 7ff5cce39640 1 --2- 172.21.15.103:0/4079629852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 0x7ff5c8110820 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.967+0000 7ff5cce39640 1 --2- 172.21.15.103:0/4079629852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5c8074e30 0x7ff5c810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.967+0000 7ff5cce39640 1 --2- 172.21.15.103:0/4079629852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5c8074510 0x7ff5c80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.967+0000 7ff5cce39640 1 -- 172.21.15.103:0/4079629852 >> 172.21.15.103:0/4079629852 conn(0x7ff5c80fd8b0 msgr2=0x7ff5c80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.967+0000 7ff5cce39640 1 -- 172.21.15.103:0/4079629852 shutdown_connections 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.967+0000 7ff5cce39640 1 -- 172.21.15.103:0/4079629852 wait complete. 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5cce39640 1 Processor -- start 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5cce39640 1 -- start start 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5cce39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5c8074510 0x7ff5c819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5cce39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5c8074e30 0x7ff5c819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5cce39640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 0x7ff5c819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5cce39640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8113ff0 con 0x7ff5c810e430 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5cce39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8113e70 con 0x7ff5c8074510 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5cce39640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8114170 con 0x7ff5c8074e30 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5c6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 0x7ff5c819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:48.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5c6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 0x7ff5c819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41930/0 (socket says 172.21.15.103:41930) 2024-09-05T01:59:48.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.968+0000 7ff5c6d9d640 1 -- 172.21.15.103:0/1004374814 learned_addr learned my addr 172.21.15.103:0/1004374814 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:48.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5c5d9b640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5c8074e30 0x7ff5c819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:48.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5c659c640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5c8074510 0x7ff5c819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:48.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5c6d9d640 1 -- 172.21.15.103:0/1004374814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5c8074e30 msgr2=0x7ff5c819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:48.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5c6d9d640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5c8074e30 0x7ff5c819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5c6d9d640 1 -- 172.21.15.103:0/1004374814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5c8074510 msgr2=0x7ff5c819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:48.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5c6d9d640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5c8074510 0x7ff5c819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5c6d9d640 1 -- 172.21.15.103:0/1004374814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c819f4a0 con 0x7ff5c810e430 2024-09-05T01:59:48.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5c659c640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5c8074510 0x7ff5c819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5c6d9d640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 0x7ff5c819edc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5c002ae70 tx=0x7ff5c002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5b77fe640 1 -- 172.21.15.103:0/1004374814 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0034040 con 0x7ff5c810e430 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c819f730 con 0x7ff5c810e430 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5b77fe640 1 -- 172.21.15.103:0/1004374814 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5c00308f0 con 0x7ff5c810e430 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5b77fe640 1 -- 172.21.15.103:0/1004374814 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c00047c0 con 0x7ff5c810e430 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.969+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c81a7070 con 0x7ff5c810e430 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.970+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c80748f0 con 0x7ff5c810e430 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.971+0000 7ff5b77fe640 1 -- 172.21.15.103:0/1004374814 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff5c0037020 con 0x7ff5c810e430 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.972+0000 7ff5b77fe640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b00a2b80 0x7ff5b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.972+0000 7ff5c659c640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b00a2b80 0x7ff5b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.972+0000 7ff5b77fe640 1 -- 172.21.15.103:0/1004374814 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7ff5c00efea0 con 0x7ff5c810e430 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.973+0000 7ff5c659c640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b00a2b80 0x7ff5b00a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009b40 tx=0x7ff5b8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:48.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:47.976+0000 7ff5b77fe640 1 -- 172.21.15.103:0/1004374814 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5c00811b0 con 0x7ff5c810e430 2024-09-05T01:59:48.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:47 smithi139 ceph-mon[34262]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:48.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.182+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5c8109610 con 0x7ff5c810e430 2024-09-05T01:59:48.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.182+0000 7ff5b77fe640 1 -- 172.21.15.103:0/1004374814 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5575 (secure 0 0 0) 0x7ff5c00bd200 con 0x7ff5c810e430 2024-09-05T01:59:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:48.183 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":7,"btime":"2024-09-05T01:59:45:637131+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-05T01:59:48.183 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 7 2024-09-05T01:59:48.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.184+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b00a2b80 msgr2=0x7ff5b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:48.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.184+0000 7ff5cce39640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b00a2b80 0x7ff5b00a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009b40 tx=0x7ff5b8005e50 comp rx=0 tx=0).stop 2024-09-05T01:59:48.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.184+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 msgr2=0x7ff5c819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:48.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.184+0000 7ff5cce39640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 0x7ff5c819edc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5c002ae70 tx=0x7ff5c002f5d0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.185+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 shutdown_connections 2024-09-05T01:59:48.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.185+0000 7ff5cce39640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5c810e430 0x7ff5c819edc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.185+0000 7ff5cce39640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff5b00a2b80 0x7ff5b00a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.185+0000 7ff5cce39640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5c8074e30 0x7ff5c819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.185+0000 7ff5cce39640 1 --2- 172.21.15.103:0/1004374814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5c8074510 0x7ff5c819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:48.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.185+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 >> 172.21.15.103:0/1004374814 conn(0x7ff5c80fd8b0 msgr2=0x7ff5c80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:48.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.185+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 shutdown_connections 2024-09-05T01:59:48.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:48.185+0000 7ff5cce39640 1 -- 172.21.15.103:0/1004374814 wait complete. 2024-09-05T01:59:48.214 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 01:59:47 smithi167 podman[66015]: 2024-09-05 01:59:47.849925729 +0000 UTC m=+0.015931404 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:59:48.215 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 01:59:47 smithi167 podman[66015]: 2024-09-05 01:59:47.969889348 +0000 UTC m=+0.135895009 container create 61d6142510966bf4bf5dcfa08757ab70f8df130d0ee53d4148e1e425f941ebaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:59:48.215 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 01:59:48 smithi167 podman[66015]: 2024-09-05 01:59:48.214545385 +0000 UTC m=+0.380551051 container init 61d6142510966bf4bf5dcfa08757ab70f8df130d0ee53d4148e1e425f941ebaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-05T01:59:48.635 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 01:59:48 smithi167 podman[66015]: 2024-09-05 01:59:48.217368646 +0000 UTC m=+0.383374305 container start 61d6142510966bf4bf5dcfa08757ab70f8df130d0ee53d4148e1e425f941ebaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-05T01:59:48.635 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 01:59:48 smithi167 ceph-mds[66034]: -- 172.21.15.167:0/2250967832 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a24825860 con 0x557a24803400 2024-09-05T01:59:48.635 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 01:59:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c[66030]: starting mds.c at 2024-09-05T01:59:48.636 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 01:59:48 smithi167 bash[66015]: 61d6142510966bf4bf5dcfa08757ab70f8df130d0ee53d4148e1e425f941ebaf 2024-09-05T01:59:48.636 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 01:59:48 smithi167 systemd[1]: Started Ceph mds.c for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:48.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:48 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1004374814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:48.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:48.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:48.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:48.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:48.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:48.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:48 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:48.982 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:48 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1004374814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:48.982 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:48.982 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:48.982 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:48.982 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:48.982 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:48.982 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:48 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:49.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:48 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1004374814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:49.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:49.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:49.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:49.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:49.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:49.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:48 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:49.979 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:49 smithi139 ceph-mon[34262]: Deploying daemon mds.h on smithi139 2024-09-05T01:59:49.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:49 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] up:boot 2024-09-05T01:59:49.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:49 smithi139 ceph-mon[34262]: fsmap 7 up:standby 2024-09-05T01:59:49.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:49 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:59:49.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:49 smithi139 ceph-mon[34262]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:50.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:49 smithi167 ceph-mon[35122]: Deploying daemon mds.h on smithi139 2024-09-05T01:59:50.346 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:49 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] up:boot 2024-09-05T01:59:50.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:49 smithi167 ceph-mon[35122]: fsmap 7 up:standby 2024-09-05T01:59:50.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:49 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:59:50.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:49 smithi167 ceph-mon[35122]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:50.347 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:49 smithi103 ceph-mon[30902]: Deploying daemon mds.h on smithi139 2024-09-05T01:59:50.348 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:49 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] up:boot 2024-09-05T01:59:50.348 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:49 smithi103 ceph-mon[30902]: fsmap 7 up:standby 2024-09-05T01:59:50.348 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:49 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:59:50.348 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:49 smithi103 ceph-mon[30902]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:50.646 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 01:59:50 smithi139 systemd[1]: Starting Ceph mds.h for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:50.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:59:50.926 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 01:59:50 smithi139 podman[64724]: 2024-09-05 01:59:50.564471563 +0000 UTC m=+0.015835622 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:59:50.926 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 01:59:50 smithi139 podman[64724]: 2024-09-05 01:59:50.681217382 +0000 UTC m=+0.132581434 container create 1fb4b4b48fb4a36ef73bf8675adc05f21826d46e9ca8f8a9d8777f10d110048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, ceph=True) 2024-09-05T01:59:50.926 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 01:59:50 smithi139 podman[64724]: 2024-09-05 01:59:50.925532018 +0000 UTC m=+0.376896072 container init 1fb4b4b48fb4a36ef73bf8675adc05f21826d46e9ca8f8a9d8777f10d110048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-05T01:59:51.113 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:51.396 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 01:59:50 smithi139 podman[64724]: 2024-09-05 01:59:50.928749391 +0000 UTC m=+0.380113447 container start 1fb4b4b48fb4a36ef73bf8675adc05f21826d46e9ca8f8a9d8777f10d110048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:59:51.396 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 01:59:50 smithi139 ceph-mds[64743]: -- 172.21.15.139:0/2886870329 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f3c22ff860 con 0x55f3c22dd400 2024-09-05T01:59:51.396 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 01:59:50 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h[64739]: starting mds.h at 2024-09-05T01:59:51.396 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 01:59:51 smithi139 bash[64724]: 1fb4b4b48fb4a36ef73bf8675adc05f21826d46e9ca8f8a9d8777f10d110048e 2024-09-05T01:59:51.396 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 01:59:51 smithi139 systemd[1]: Started Ceph mds.h for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:51.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.600+0000 7f5db23f6640 1 -- 172.21.15.103:0/1437647125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac1036c0 msgr2=0x7f5dac10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.600+0000 7f5db23f6640 1 --2- 172.21.15.103:0/1437647125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac1036c0 0x7f5dac10bbb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009900 tx=0x7f5da002a480 comp rx=0 tx=0).stop 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.600+0000 7f5db23f6640 1 -- 172.21.15.103:0/1437647125 shutdown_connections 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.600+0000 7f5db23f6640 1 --2- 172.21.15.103:0/1437647125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dac10c180 0x7f5dac10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.600+0000 7f5db23f6640 1 --2- 172.21.15.103:0/1437647125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac1036c0 0x7f5dac10bbb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.600+0000 7f5db23f6640 1 --2- 172.21.15.103:0/1437647125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac102da0 0x7f5dac103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.600+0000 7f5db23f6640 1 -- 172.21.15.103:0/1437647125 >> 172.21.15.103:0/1437647125 conn(0x7f5dac0fc7a0 msgr2=0x7f5dac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.600+0000 7f5db23f6640 1 -- 172.21.15.103:0/1437647125 shutdown_connections 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.600+0000 7f5db23f6640 1 -- 172.21.15.103:0/1437647125 wait complete. 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5db23f6640 1 Processor -- start 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5db23f6640 1 -- start start 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5db23f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac102da0 0x7f5dac198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5db23f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dac1036c0 0x7f5dac198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5db23f6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac10c180 0x7f5dac19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5db23f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac111e10 con 0x7f5dac102da0 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5db23f6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac111c90 con 0x7f5dac1036c0 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5db23f6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac111f90 con 0x7f5dac10c180 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5dabfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac102da0 0x7f5dac198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5dabfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac102da0 0x7f5dac198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53026/0 (socket says 172.21.15.103:53026) 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5dabfff640 1 -- 172.21.15.103:0/2765830753 learned_addr learned my addr 172.21.15.103:0/2765830753 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5db0993640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac10c180 0x7f5dac19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.601+0000 7f5dab7fe640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dac1036c0 0x7f5dac198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5dabfff640 1 -- 172.21.15.103:0/2765830753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac10c180 msgr2=0x7f5dac19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5dabfff640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac10c180 0x7f5dac19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5dabfff640 1 -- 172.21.15.103:0/2765830753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dac1036c0 msgr2=0x7f5dac198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5dabfff640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dac1036c0 0x7f5dac198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5dabfff640 1 -- 172.21.15.103:0/2765830753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac19d2c0 con 0x7f5dac102da0 2024-09-05T01:59:51.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5dab7fe640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dac1036c0 0x7f5dac198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5dabfff640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac102da0 0x7f5dac198310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c970 tx=0x7f5d9c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5da97fa640 1 -- 172.21.15.103:0/2765830753 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c007bb0 con 0x7f5dac102da0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac19d5b0 con 0x7f5dac102da0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5da97fa640 1 -- 172.21.15.103:0/2765830753 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d9c007d50 con 0x7f5dac102da0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5da97fa640 1 -- 172.21.15.103:0/2765830753 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00f450 con 0x7f5dac102da0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.602+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac1a4e90 con 0x7f5dac102da0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.603+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac103180 con 0x7f5dac102da0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.604+0000 7f5da97fa640 1 -- 172.21.15.103:0/2765830753 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5d9c004f20 con 0x7f5dac102da0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.605+0000 7f5da97fa640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5d940a2b80 0x7f5d940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.605+0000 7f5dab7fe640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5d940a2b80 0x7f5d940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.605+0000 7f5da97fa640 1 -- 172.21.15.103:0/2765830753 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f5d9c0ce920 con 0x7f5dac102da0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.605+0000 7f5dab7fe640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5d940a2b80 0x7f5d940a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5da0031440 tx=0x7f5da00095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:51.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.609+0000 7f5da97fa640 1 -- 172.21.15.103:0/2765830753 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d9c014030 con 0x7f5dac102da0 2024-09-05T01:59:51.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.811+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dac1071c0 con 0x7f5dac102da0 2024-09-05T01:59:51.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.812+0000 7f5da97fa640 1 -- 172.21.15.103:0/2765830753 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7231 (secure 0 0 0) 0x7f5d9c09bd20 con 0x7f5dac102da0 2024-09-05T01:59:51.813 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":9,"btime":"2024-09-05T01:59:51:007076+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-05T01:59:51.813 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 9 2024-09-05T01:59:51.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5d940a2b80 msgr2=0x7f5d940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:51.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5d940a2b80 0x7f5d940a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5da0031440 tx=0x7f5da00095f0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac102da0 msgr2=0x7f5dac198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:51.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac102da0 0x7f5dac198310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c970 tx=0x7f5d9c00ce30 comp rx=0 tx=0).stop 2024-09-05T01:59:51.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 shutdown_connections 2024-09-05T01:59:51.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5d940a2b80 0x7f5d940a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac10c180 0x7f5dac19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dac1036c0 0x7f5dac198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 --2- 172.21.15.103:0/2765830753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dac102da0 0x7f5dac198310 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:51.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 >> 172.21.15.103:0/2765830753 conn(0x7f5dac0fc7a0 msgr2=0x7f5dac0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:51.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 shutdown_connections 2024-09-05T01:59:51.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:51.814+0000 7f5db23f6640 1 -- 172.21.15.103:0/2765830753 wait complete. 2024-09-05T01:59:52.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] up:boot 2024-09-05T01:59:52.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: fsmap 8 up:standby 2024-09-05T01:59:52.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T01:59:52.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:52.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:52.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:52.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:52.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:52.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:52.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:52.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: Deploying daemon mds.i on smithi167 2024-09-05T01:59:52.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:52 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2765830753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:52.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] up:boot 2024-09-05T01:59:52.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: fsmap 8 up:standby 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: Deploying daemon mds.i on smithi167 2024-09-05T01:59:52.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:52 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2765830753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] up:boot 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: fsmap 8 up:standby 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:52.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:52.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: Deploying daemon mds.i on smithi167 2024-09-05T01:59:52.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:52 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2765830753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:53.135 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 01:59:53 smithi167 systemd[1]: Starting Ceph mds.i for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:53.529 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 01:59:53 smithi167 podman[66316]: 2024-09-05 01:59:53.160560711 +0000 UTC m=+0.015643456 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:59:53.530 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 01:59:53 smithi167 podman[66316]: 2024-09-05 01:59:53.276574951 +0000 UTC m=+0.131657689 container create 6155c037fa44f7d63023dc796e53497ff3c925a241e2fb9b552f76e2e60673d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-05T01:59:53.530 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 01:59:53 smithi167 podman[66316]: 2024-09-05 01:59:53.529354314 +0000 UTC m=+0.384437051 container init 6155c037fa44f7d63023dc796e53497ff3c925a241e2fb9b552f76e2e60673d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-05T01:59:53.885 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 01:59:53 smithi167 podman[66316]: 2024-09-05 01:59:53.532333073 +0000 UTC m=+0.387415810 container start 6155c037fa44f7d63023dc796e53497ff3c925a241e2fb9b552f76e2e60673d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-05T01:59:53.885 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 01:59:53 smithi167 ceph-mds[66336]: -- 172.21.15.167:0/1605303373 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d15a89860 con 0x563d15a67000 2024-09-05T01:59:53.885 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 01:59:53 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i[66332]: starting mds.i at 2024-09-05T01:59:53.886 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 01:59:53 smithi167 bash[66316]: 6155c037fa44f7d63023dc796e53497ff3c925a241e2fb9b552f76e2e60673d4 2024-09-05T01:59:53.886 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 01:59:53 smithi167 systemd[1]: Started Ceph mds.i for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:54.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:54 smithi103 ceph-mon[30902]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:54.396 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:54.396 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:54.396 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:54.396 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:54.396 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:54.396 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:54 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:54.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:54 smithi139 ceph-mon[34262]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:54.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:54 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:54.539 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:59:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:54 smithi167 ceph-mon[35122]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:54.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:54.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:54.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:54.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:54.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:54.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:54 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:55.393 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:55 smithi103 ceph-mon[30902]: Deploying daemon mds.g on smithi103 2024-09-05T01:59:55.394 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:55 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] up:boot 2024-09-05T01:59:55.394 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:55 smithi103 ceph-mon[30902]: fsmap 9 up:standby 2024-09-05T01:59:55.394 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:55 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T01:59:55.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:55 smithi139 ceph-mon[34262]: Deploying daemon mds.g on smithi103 2024-09-05T01:59:55.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:55 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] up:boot 2024-09-05T01:59:55.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:55 smithi139 ceph-mon[34262]: fsmap 9 up:standby 2024-09-05T01:59:55.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:55 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T01:59:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:55 smithi167 ceph-mon[35122]: Deploying daemon mds.g on smithi103 2024-09-05T01:59:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:55 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] up:boot 2024-09-05T01:59:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:55 smithi167 ceph-mon[35122]: fsmap 9 up:standby 2024-09-05T01:59:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:55 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T01:59:56.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.988+0000 7f8f8ce44640 1 -- 172.21.15.103:0/4127016832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 msgr2=0x7f8f88106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:56.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.988+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/4127016832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 0x7f8f88106e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009930 tx=0x7f8f7802a420 comp rx=0 tx=0).stop 2024-09-05T01:59:56.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.988+0000 7f8f8ce44640 1 -- 172.21.15.103:0/4127016832 shutdown_connections 2024-09-05T01:59:56.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.988+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/4127016832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f88101420 0x7f8f8810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.988+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/4127016832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f88100a80 0x7f8f88100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.988+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/4127016832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 0x7f8f88106e60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.988+0000 7f8f8ce44640 1 -- 172.21.15.103:0/4127016832 >> 172.21.15.103:0/4127016832 conn(0x7f8f880fc7a0 msgr2=0x7f8f880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.988+0000 7f8f8ce44640 1 -- 172.21.15.103:0/4127016832 shutdown_connections 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.988+0000 7f8f8ce44640 1 -- 172.21.15.103:0/4127016832 wait complete. 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f8ce44640 1 Processor -- start 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f8ce44640 1 -- start start 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f8ce44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f88100a80 0x7f8f881982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f8ce44640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f88101420 0x7f8f881987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f8ce44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 0x7f8f8819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f8ce44640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88111e50 con 0x7f8f88106a80 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f8ce44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88111cd0 con 0x7f8f88100a80 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f8ce44640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88111fd0 con 0x7f8f88101420 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f86d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 0x7f8f8819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f86d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 0x7f8f8819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53050/0 (socket says 172.21.15.103:53050) 2024-09-05T01:59:56.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f86d9d640 1 -- 172.21.15.103:0/2201737738 learned_addr learned my addr 172.21.15.103:0/2201737738 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f85d9b640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f88101420 0x7f8f881987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.989+0000 7f8f8659c640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f88100a80 0x7f8f881982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f86d9d640 1 -- 172.21.15.103:0/2201737738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f88101420 msgr2=0x7f8f881987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f86d9d640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f88101420 0x7f8f881987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f86d9d640 1 -- 172.21.15.103:0/2201737738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f88100a80 msgr2=0x7f8f881982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f86d9d640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f88100a80 0x7f8f881982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f86d9d640 1 -- 172.21.15.103:0/2201737738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8819d260 con 0x7f8f88106a80 2024-09-05T01:59:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f8659c640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f88100a80 0x7f8f881982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:59:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f86d9d640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 0x7f8f8819cb80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000cce0 tx=0x7f8f8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f7f7fe640 1 -- 172.21.15.103:0/2201737738 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8000f450 con 0x7f8f88106a80 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8819d550 con 0x7f8f88106a80 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f7f7fe640 1 -- 172.21.15.103:0/2201737738 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f8000faa0 con 0x7f8f88106a80 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f7f7fe640 1 -- 172.21.15.103:0/2201737738 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80018a80 con 0x7f8f88106a80 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.990+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f881a4e30 con 0x7f8f88106a80 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.991+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f88100ee0 con 0x7f8f88106a80 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.992+0000 7f8f7f7fe640 1 -- 172.21.15.103:0/2201737738 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8f8000fc50 con 0x7f8f88106a80 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.992+0000 7f8f7f7fe640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8f740a2b80 0x7f8f740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.993+0000 7f8f8659c640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8f740a2b80 0x7f8f740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.993+0000 7f8f7f7fe640 1 -- 172.21.15.103:0/2201737738 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f8f800cefd0 con 0x7f8f88106a80 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.993+0000 7f8f8659c640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8f740a2b80 0x7f8f740a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f78000c00 tx=0x7f8f78031cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:55.997+0000 7f8f7f7fe640 1 -- 172.21.15.103:0/2201737738 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f80014030 con 0x7f8f88106a80 2024-09-05T01:59:56.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.210+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f8802d7c0 con 0x7f8f88106a80 2024-09-05T01:59:56.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.211+0000 7f8f7f7fe640 1 -- 172.21.15.103:0/2201737738 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8061 (secure 0 0 0) 0x7f8f8009c3d0 con 0x7f8f88106a80 2024-09-05T01:59:56.213 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T01:59:56.213 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":10,"btime":"2024-09-05T01:59:54:128183+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-05T01:59:56.214 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 10 2024-09-05T01:59:56.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8f740a2b80 msgr2=0x7f8f740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:56.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8f740a2b80 0x7f8f740a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f78000c00 tx=0x7f8f78031cf0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 msgr2=0x7f8f8819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:56.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 0x7f8f8819cb80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000cce0 tx=0x7f8f8000a620 comp rx=0 tx=0).stop 2024-09-05T01:59:56.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 shutdown_connections 2024-09-05T01:59:56.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f88106a80 0x7f8f8819cb80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8f740a2b80 0x7f8f740a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f88101420 0x7f8f881987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 --2- 172.21.15.103:0/2201737738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f88100a80 0x7f8f881982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:56.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 >> 172.21.15.103:0/2201737738 conn(0x7f8f880fc7a0 msgr2=0x7f8f880fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:56.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 shutdown_connections 2024-09-05T01:59:56.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:56.213+0000 7f8f8ce44640 1 -- 172.21.15.103:0/2201737738 wait complete. 2024-09-05T01:59:56.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:56 smithi139 ceph-mon[34262]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:56.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:56 smithi103 ceph-mon[30902]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:56 smithi167 ceph-mon[35122]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:57.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:57 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2201737738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:57.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:57 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2201737738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:57.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:57 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2201737738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:59:57.648 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 01:59:57 smithi103 systemd[1]: Started Ceph mds.g for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T01:59:58.528 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:58 smithi103 ceph-mon[30902]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:58.528 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:58 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:58.528 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:58 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:58.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:58 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:58.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:58 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:58.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:58 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:58.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:58 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:58 smithi167 ceph-mon[35122]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:58 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:58 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:58 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:58 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:58 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:58 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:58 smithi139 ceph-mon[34262]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:59:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:58 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:58 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:58 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T01:59:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:58 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:59:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:58 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:59:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:58 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:59:58.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:59:59.320 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:59:59.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:59 smithi103 ceph-mon[30902]: Deploying daemon mds.a on smithi103 2024-09-05T01:59:59.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:59 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.103:6838/1603518036,v1:172.21.15.103:6839/1603518036] up:boot 2024-09-05T01:59:59.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:59 smithi103 ceph-mon[30902]: fsmap 10 up:standby 2024-09-05T01:59:59.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 01:59:59 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T01:59:59.399 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 01:59:59 smithi103 systemd[1]: Starting Ceph mds.a for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T01:59:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:59 smithi167 ceph-mon[35122]: Deploying daemon mds.a on smithi103 2024-09-05T01:59:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:59 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.103:6838/1603518036,v1:172.21.15.103:6839/1603518036] up:boot 2024-09-05T01:59:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:59 smithi167 ceph-mon[35122]: fsmap 10 up:standby 2024-09-05T01:59:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 01:59:59 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T01:59:59.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:59 smithi139 ceph-mon[34262]: Deploying daemon mds.a on smithi103 2024-09-05T01:59:59.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:59 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.103:6838/1603518036,v1:172.21.15.103:6839/1603518036] up:boot 2024-09-05T01:59:59.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:59 smithi139 ceph-mon[34262]: fsmap 10 up:standby 2024-09-05T01:59:59.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 01:59:59 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 -- 172.21.15.103:0/900476695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 msgr2=0x7f345810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 --2- 172.21.15.103:0/900476695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 0x7f345810d950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3450009cd0 tx=0x7f345002a8e0 comp rx=0 tx=0).stop 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 -- 172.21.15.103:0/900476695 shutdown_connections 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 --2- 172.21.15.103:0/900476695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 0x7f345810d950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 --2- 172.21.15.103:0/900476695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3458100a80 0x7f3458100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 --2- 172.21.15.103:0/900476695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3458106a80 0x7f3458106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 -- 172.21.15.103:0/900476695 >> 172.21.15.103:0/900476695 conn(0x7f34580fc7a0 msgr2=0x7f34580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 -- 172.21.15.103:0/900476695 shutdown_connections 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 -- 172.21.15.103:0/900476695 wait complete. 2024-09-05T01:59:59.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 Processor -- start 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.801+0000 7f345e968640 1 -- start start 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f345e968640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3458100a80 0x7f34581972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f345e968640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 0x7f3458197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f345e968640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3458106a80 0x7f345819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f345e968640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3458110d50 con 0x7f3458101420 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f345e968640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3458110bd0 con 0x7f3458106a80 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f345e968640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3458110ed0 con 0x7f3458100a80 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f34577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 0x7f3458197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f34577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 0x7f3458197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60858/0 (socket says 172.21.15.103:60858) 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f34577fe640 1 -- 172.21.15.103:0/3219052969 learned_addr learned my addr 172.21.15.103:0/3219052969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f3457fff640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3458100a80 0x7f34581972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f345cf05640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3458106a80 0x7f345819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f34577fe640 1 -- 172.21.15.103:0/3219052969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3458100a80 msgr2=0x7f34581972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f34577fe640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3458100a80 0x7f34581972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:59.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f34577fe640 1 -- 172.21.15.103:0/3219052969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3458106a80 msgr2=0x7f345819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f34577fe640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3458106a80 0x7f345819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f34577fe640 1 -- 172.21.15.103:0/3219052969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345819c2a0 con 0x7f3458101420 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.802+0000 7f34577fe640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 0x7f3458197830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f344c00d990 tx=0x7f344c00de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.803+0000 7f34557fa640 1 -- 172.21.15.103:0/3219052969 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c01b070 con 0x7f3458101420 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.803+0000 7f34557fa640 1 -- 172.21.15.103:0/3219052969 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f344c00edf0 con 0x7f3458101420 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.803+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345819c590 con 0x7f3458101420 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.803+0000 7f34557fa640 1 -- 172.21.15.103:0/3219052969 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c014800 con 0x7f3458101420 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.803+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345806a790 con 0x7f3458101420 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.804+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34581021c0 con 0x7f3458101420 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.805+0000 7f34557fa640 1 -- 172.21.15.103:0/3219052969 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f344c01e070 con 0x7f3458101420 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.805+0000 7f34557fa640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f34400a2b80 0x7f34400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.806+0000 7f3457fff640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f34400a2b80 0x7f34400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:59:59.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.806+0000 7f34557fa640 1 -- 172.21.15.103:0/3219052969 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f344c0d0330 con 0x7f3458101420 2024-09-05T01:59:59.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.806+0000 7f3457fff640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f34400a2b80 0x7f34400a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3448005fd0 tx=0x7f3448005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:59:59.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T01:59:59.810+0000 7f34557fa640 1 -- 172.21.15.103:0/3219052969 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f344c09d730 con 0x7f3458101420 2024-09-05T02:00:00.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.018+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3458100ee0 con 0x7f3458101420 2024-09-05T02:00:00.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.019+0000 7f34557fa640 1 -- 172.21.15.103:0/3219052969 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8890 (secure 0 0 0) 0x7f3458100ee0 con 0x7f3458101420 2024-09-05T02:00:00.020 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:00.021 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":11,"btime":"2024-09-05T01:59:58:185686+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-05T02:00:00.021 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 11 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f34400a2b80 msgr2=0x7f34400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f34400a2b80 0x7f34400a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3448005fd0 tx=0x7f3448005850 comp rx=0 tx=0).stop 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 msgr2=0x7f3458197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 0x7f3458197830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f344c00d990 tx=0x7f344c00de50 comp rx=0 tx=0).stop 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 shutdown_connections 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3458106a80 0x7f345819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3458101420 0x7f3458197830 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f34400a2b80 0x7f34400a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 --2- 172.21.15.103:0/3219052969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3458100a80 0x7f34581972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 >> 172.21.15.103:0/3219052969 conn(0x7f34580fc7a0 msgr2=0x7f34580fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:00.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 shutdown_connections 2024-09-05T02:00:00.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:00.021+0000 7f345e968640 1 -- 172.21.15.103:0/3219052969 wait complete. 2024-09-05T02:00:00.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:00 smithi103 ceph-mon[30902]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:00.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:00 smithi103 ceph-mon[30902]: overall HEALTH_OK 2024-09-05T02:00:00.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:00 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3219052969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:00.399 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 01:59:59 smithi103 podman[69057]: 2024-09-05 01:59:59.887023231 +0000 UTC m=+0.017008517 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:00:00.400 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:00:00 smithi103 podman[69057]: 2024-09-05 02:00:00.037030626 +0000 UTC m=+0.167015913 container create 9a2f9e3e93e8e23a9d0ac44ff8e8143040680eed7819425ae4d2edb8c3142449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-05T02:00:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:00 smithi167 ceph-mon[35122]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:00 smithi167 ceph-mon[35122]: overall HEALTH_OK 2024-09-05T02:00:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:00 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3219052969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:00 smithi139 ceph-mon[34262]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:00 smithi139 ceph-mon[34262]: overall HEALTH_OK 2024-09-05T02:00:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:00 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3219052969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:01.148 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:00:00 smithi103 podman[69057]: 2024-09-05 02:00:00.929492535 +0000 UTC m=+1.059477836 container init 9a2f9e3e93e8e23a9d0ac44ff8e8143040680eed7819425ae4d2edb8c3142449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-91c8a7e) 2024-09-05T02:00:01.148 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:00:00 smithi103 podman[69057]: 2024-09-05 02:00:00.932687516 +0000 UTC m=+1.062672803 container start 9a2f9e3e93e8e23a9d0ac44ff8e8143040680eed7819425ae4d2edb8c3142449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-05T02:00:01.149 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:00:00 smithi103 ceph-mds[69093]: -- 172.21.15.103:0/3357076153 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56439dc4d860 con 0x56439dc2b400 2024-09-05T02:00:01.149 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:00:00 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a[69086]: starting mds.a at 2024-09-05T02:00:01.149 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:00:01 smithi103 bash[69057]: 9a2f9e3e93e8e23a9d0ac44ff8e8143040680eed7819425ae4d2edb8c3142449 2024-09-05T02:00:01.149 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:00:01 smithi103 systemd[1]: Started Ceph mds.a for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T02:00:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:02.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T02:00:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T02:00:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:00:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: Deploying daemon mds.b on smithi139 2024-09-05T02:00:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.103:6840/426074448,v1:172.21.15.103:6841/426074448] up:boot 2024-09-05T02:00:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: fsmap 11 up:standby 2024-09-05T02:00:02.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:02 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: Deploying daemon mds.b on smithi139 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.103:6840/426074448,v1:172.21.15.103:6841/426074448] up:boot 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: fsmap 11 up:standby 2024-09-05T02:00:02.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:02 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T02:00:02.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:02.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:02.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T02:00:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T02:00:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:00:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: Deploying daemon mds.b on smithi139 2024-09-05T02:00:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.103:6840/426074448,v1:172.21.15.103:6841/426074448] up:boot 2024-09-05T02:00:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: fsmap 11 up:standby 2024-09-05T02:00:02.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:02 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T02:00:02.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:00:03.079 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:03.396 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:03 smithi139 systemd[1]: Starting Ceph mds.b for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:00:03.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.574+0000 7f30de14a640 1 -- 172.21.15.103:0/276805579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8100a90 msgr2=0x7f30d8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:03.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.574+0000 7f30de14a640 1 --2- 172.21.15.103:0/276805579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8100a90 0x7f30d8100ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009930 tx=0x7f30c002a420 comp rx=0 tx=0).stop 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.574+0000 7f30de14a640 1 -- 172.21.15.103:0/276805579 shutdown_connections 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.574+0000 7f30de14a640 1 --2- 172.21.15.103:0/276805579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d8101430 0x7f30d810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.574+0000 7f30de14a640 1 --2- 172.21.15.103:0/276805579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8100a90 0x7f30d8100ef0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.574+0000 7f30de14a640 1 --2- 172.21.15.103:0/276805579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d8106a90 0x7f30d8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.574+0000 7f30de14a640 1 -- 172.21.15.103:0/276805579 >> 172.21.15.103:0/276805579 conn(0x7f30d80fc7d0 msgr2=0x7f30d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.574+0000 7f30de14a640 1 -- 172.21.15.103:0/276805579 shutdown_connections 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.574+0000 7f30de14a640 1 -- 172.21.15.103:0/276805579 wait complete. 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30de14a640 1 Processor -- start 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30de14a640 1 -- start start 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30de14a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d8100a90 0x7f30d819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30de14a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8101430 0x7f30d819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30de14a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d8106a90 0x7f30d819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30de14a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8110d40 con 0x7f30d8101430 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30de14a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8110bc0 con 0x7f30d8106a90 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30de14a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8110ec0 con 0x7f30d8100a90 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d8106a90 0x7f30d819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30d77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d8100a90 0x7f30d819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8101430 0x7f30d819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d8106a90 0x7f30d819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:45090/0 (socket says 172.21.15.103:45090) 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30d77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d8100a90 0x7f30d819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:36480/0 (socket says 172.21.15.103:36480) 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8101430 0x7f30d819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60884/0 (socket says 172.21.15.103:60884) 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.575+0000 7f30d7fff640 1 -- 172.21.15.103:0/4043300551 learned_addr learned my addr 172.21.15.103:0/4043300551 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30d7fff640 1 -- 172.21.15.103:0/4043300551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d8100a90 msgr2=0x7f30d819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30d7fff640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d8100a90 0x7f30d819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30d7fff640 1 -- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8101430 msgr2=0x7f30d819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:03.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30d7fff640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8101430 0x7f30d819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30d7fff640 1 -- 172.21.15.103:0/4043300551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d819f590 con 0x7f30d8106a90 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30cffff640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8101430 0x7f30d819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30d7fff640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d8106a90 0x7f30d819eeb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30d000efd0 tx=0x7f30d000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30d57fa640 1 -- 172.21.15.103:0/4043300551 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0003b70 con 0x7f30d8106a90 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30d57fa640 1 -- 172.21.15.103:0/4043300551 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d0003d10 con 0x7f30d8106a90 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d819f880 con 0x7f30d8106a90 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30d57fa640 1 -- 172.21.15.103:0/4043300551 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0010980 con 0x7f30d8106a90 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.576+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d81a7160 con 0x7f30d8106a90 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.577+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d8100ef0 con 0x7f30d8106a90 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.579+0000 7f30d57fa640 1 -- 172.21.15.103:0/4043300551 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f30d0004b40 con 0x7f30d8106a90 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.579+0000 7f30d57fa640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f30c40a2b80 0x7f30c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.580+0000 7f30d77fe640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f30c40a2b80 0x7f30c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:03.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.580+0000 7f30d57fa640 1 -- 172.21.15.103:0/4043300551 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f30d00cf180 con 0x7f30d8106a90 2024-09-05T02:00:03.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.580+0000 7f30d77fe640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f30c40a2b80 0x7f30c40a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f30c80058c0 tx=0x7f30c800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:03.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.582+0000 7f30d57fa640 1 -- 172.21.15.103:0/4043300551 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30d009c580 con 0x7f30d8106a90 2024-09-05T02:00:03.654 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:03 smithi139 podman[65025]: 2024-09-05 02:00:03.305228828 +0000 UTC m=+0.015422708 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:00:03.654 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:03 smithi139 podman[65025]: 2024-09-05 02:00:03.417160803 +0000 UTC m=+0.127354685 container create 458b198b21b42a63883cdcbe8a971b9a688fa686d71da877b6cae79b7de18021 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-05T02:00:03.654 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:03 smithi139 podman[65025]: 2024-09-05 02:00:03.653617835 +0000 UTC m=+0.363811720 container init 458b198b21b42a63883cdcbe8a971b9a688fa686d71da877b6cae79b7de18021 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e) 2024-09-05T02:00:03.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.790+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30d81a73c0 con 0x7f30d8106a90 2024-09-05T02:00:03.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.790+0000 7f30d57fa640 1 -- 172.21.15.103:0/4043300551 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9716 (secure 0 0 0) 0x7f30d009be80 con 0x7f30d8106a90 2024-09-05T02:00:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:03.793 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 12 2024-09-05T02:00:03.793 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":12,"btime":"2024-09-05T02:00:01:198579+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-05T02:00:03.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f30c40a2b80 msgr2=0x7f30c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:03.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f30c40a2b80 0x7f30c40a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f30c80058c0 tx=0x7f30c800a250 comp rx=0 tx=0).stop 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d8106a90 msgr2=0x7f30d819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d8106a90 0x7f30d819eeb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30d000efd0 tx=0x7f30d000c6a0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 shutdown_connections 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d8106a90 0x7f30d819eeb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30d8101430 0x7f30d819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f30c40a2b80 0x7f30c40a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 --2- 172.21.15.103:0/4043300551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d8100a90 0x7f30d819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 >> 172.21.15.103:0/4043300551 conn(0x7f30d80fc7d0 msgr2=0x7f30d806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 shutdown_connections 2024-09-05T02:00:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:03.793+0000 7f30de14a640 1 -- 172.21.15.103:0/4043300551 wait complete. 2024-09-05T02:00:04.086 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:03 smithi139 podman[65025]: 2024-09-05 02:00:03.657016246 +0000 UTC m=+0.367210137 container start 458b198b21b42a63883cdcbe8a971b9a688fa686d71da877b6cae79b7de18021 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-05T02:00:04.086 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:03 smithi139 ceph-mds[65044]: -- 172.21.15.139:0/4207360016 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ea59e5860 con 0x564ea59c3000 2024-09-05T02:00:04.086 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b[65040]: starting mds.b at 2024-09-05T02:00:04.086 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:03 smithi139 bash[65025]: 458b198b21b42a63883cdcbe8a971b9a688fa686d71da877b6cae79b7de18021 2024-09-05T02:00:04.086 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:03 smithi139 systemd[1]: Started Ceph mds.b for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T02:00:04.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:04 smithi139 ceph-mon[34262]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:04.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:04 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/4043300551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:04.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:04 smithi167 ceph-mon[35122]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:04.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:04 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/4043300551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:04.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:04 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:00:04.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:04 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:00:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:04 smithi103 ceph-mon[30902]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:04 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/4043300551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:04.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:04 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:00:05.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:05 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] up:boot 2024-09-05T02:00:05.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:05 smithi103 ceph-mon[30902]: fsmap 12 up:standby 2024-09-05T02:00:05.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T02:00:05.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:05 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:05.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:05 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] up:boot 2024-09-05T02:00:05.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:05 smithi167 ceph-mon[35122]: fsmap 12 up:standby 2024-09-05T02:00:05.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T02:00:05.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:05 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:05.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:05 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] up:boot 2024-09-05T02:00:05.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:05 smithi139 ceph-mon[34262]: fsmap 12 up:standby 2024-09-05T02:00:05.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T02:00:05.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:05 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:06.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:00:06.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:00:06.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:06 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:00:06.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:06 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:00:06.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:06.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:00:06.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:00:06.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:06.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:06 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:00:06.512 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:00:07.111 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:07.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 -- 172.21.15.103:0/3710381218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 msgr2=0x7fc3b81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:07.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 --2- 172.21.15.103:0/3710381218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 0x7fc3b81029b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009900 tx=0x7fc3a802a480 comp rx=0 tx=0).stop 2024-09-05T02:00:07.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 -- 172.21.15.103:0/3710381218 shutdown_connections 2024-09-05T02:00:07.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 --2- 172.21.15.103:0/3710381218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3b8102ef0 0x7fc3b810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 --2- 172.21.15.103:0/3710381218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 0x7fc3b81029b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 --2- 172.21.15.103:0/3710381218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3b8100e20 0x7fc3b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 -- 172.21.15.103:0/3710381218 >> 172.21.15.103:0/3710381218 conn(0x7fc3b80fc7d0 msgr2=0x7fc3b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 -- 172.21.15.103:0/3710381218 shutdown_connections 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 -- 172.21.15.103:0/3710381218 wait complete. 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 Processor -- start 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.628+0000 7fc3c02da640 1 -- start start 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3c02da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3b8100e20 0x7fc3b8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3c02da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 0x7fc3b81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3c02da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3b8102ef0 0x7fc3b81a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3c02da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8110e30 con 0x7fc3b8102570 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3c02da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8110cb0 con 0x7fc3b8102ef0 2024-09-05T02:00:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3c02da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8110fb0 con 0x7fc3b8100e20 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3bd875640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 0x7fc3b81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3bd875640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 0x7fc3b81997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60902/0 (socket says 172.21.15.103:60902) 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3bd875640 1 -- 172.21.15.103:0/2142591048 learned_addr learned my addr 172.21.15.103:0/2142591048 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3be877640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3b8102ef0 0x7fc3b81a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3bd875640 1 -- 172.21.15.103:0/2142591048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3b8100e20 msgr2=0x7fc3b8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3bd875640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3b8100e20 0x7fc3b8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3bd875640 1 -- 172.21.15.103:0/2142591048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3b8102ef0 msgr2=0x7fc3b81a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3bd875640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3b8102ef0 0x7fc3b81a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3bd875640 1 -- 172.21.15.103:0/2142591048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b81a4820 con 0x7fc3b8102570 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3be877640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3b8102ef0 0x7fc3b81a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3bd875640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 0x7fc3b81997a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8006010 tx=0x7fc3a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:07.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3af7fe640 1 -- 172.21.15.103:0/2142591048 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a803f030 con 0x7fc3b8102570 2024-09-05T02:00:07.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3af7fe640 1 -- 172.21.15.103:0/2142591048 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3a8003cb0 con 0x7fc3b8102570 2024-09-05T02:00:07.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b81a49f0 con 0x7fc3b8102570 2024-09-05T02:00:07.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3af7fe640 1 -- 172.21.15.103:0/2142591048 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8035070 con 0x7fc3b8102570 2024-09-05T02:00:07.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.629+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b81a4ea0 con 0x7fc3b8102570 2024-09-05T02:00:07.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.630+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b8101200 con 0x7fc3b8102570 2024-09-05T02:00:07.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.631+0000 7fc3af7fe640 1 -- 172.21.15.103:0/2142591048 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc3a8042030 con 0x7fc3b8102570 2024-09-05T02:00:07.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.632+0000 7fc3af7fe640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc3a40a2c00 0x7fc3a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:07.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.632+0000 7fc3af7fe640 1 -- 172.21.15.103:0/2142591048 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc3a80ef0b0 con 0x7fc3b8102570 2024-09-05T02:00:07.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.632+0000 7fc3be076640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc3a40a2c00 0x7fc3a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:07.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.633+0000 7fc3be076640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc3a40a2c00 0x7fc3a40a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4005fd0 tx=0x7fc3b4005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:07.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.636+0000 7fc3af7fe640 1 -- 172.21.15.103:0/2142591048 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3a80bc4b0 con 0x7fc3b8102570 2024-09-05T02:00:07.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.865+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3b8109090 con 0x7fc3b8102570 2024-09-05T02:00:07.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.866+0000 7fc3af7fe640 1 -- 172.21.15.103:0/2142591048 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10545 (secure 0 0 0) 0x7fc3a80bbdb0 con 0x7fc3b8102570 2024-09-05T02:00:07.869 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:07.869 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":13,"btime":"2024-09-05T02:00:04:082330+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-05T02:00:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 13 2024-09-05T02:00:07.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.868+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc3a40a2c00 msgr2=0x7fc3a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:07.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.868+0000 7fc3c02da640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc3a40a2c00 0x7fc3a40a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4005fd0 tx=0x7fc3b4005c70 comp rx=0 tx=0).stop 2024-09-05T02:00:07.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.868+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 msgr2=0x7fc3b81997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:07.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.868+0000 7fc3c02da640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 0x7fc3b81997a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8006010 tx=0x7fc3a80042f0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.869+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 shutdown_connections 2024-09-05T02:00:07.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.869+0000 7fc3c02da640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3b8102ef0 0x7fc3b81a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.869+0000 7fc3c02da640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8102570 0x7fc3b81997a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.869+0000 7fc3c02da640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc3a40a2c00 0x7fc3a40a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.869+0000 7fc3c02da640 1 --2- 172.21.15.103:0/2142591048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3b8100e20 0x7fc3b8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:07.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.869+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 >> 172.21.15.103:0/2142591048 conn(0x7fc3b80fc7d0 msgr2=0x7fc3b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:07.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.869+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 shutdown_connections 2024-09-05T02:00:07.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:07.869+0000 7fc3c02da640 1 -- 172.21.15.103:0/2142591048 wait complete. 2024-09-05T02:00:08.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:08 smithi103 ceph-mon[30902]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:08.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:08 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:08 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:08 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:08 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:08 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2142591048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:08.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:08 smithi167 ceph-mon[35122]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:08.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:08 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:08 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:08 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:08 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:08 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2142591048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:08.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:08 smithi139 ceph-mon[34262]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:08.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:08 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:08 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:08 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:08 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:08.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:08 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2142591048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:08.524 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-05T02:00:08.772 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:09.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:09 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:09.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:09 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:09.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:09 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:00:09.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:09 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:00:09.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:09 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:09.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.262+0000 7f92aea5c640 1 -- 172.21.15.103:0/928635775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8106f10 msgr2=0x7f92a810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:09.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.262+0000 7f92aea5c640 1 --2- 172.21.15.103:0/928635775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8106f10 0x7f92a810ea90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009ca0 tx=0x7f92a002a8c0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.262+0000 7f92aea5c640 1 -- 172.21.15.103:0/928635775 shutdown_connections 2024-09-05T02:00:09.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.262+0000 7f92aea5c640 1 --2- 172.21.15.103:0/928635775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8106f10 0x7f92a810ea90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.262+0000 7f92aea5c640 1 --2- 172.21.15.103:0/928635775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a8102790 0x7f92a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.262+0000 7f92aea5c640 1 --2- 172.21.15.103:0/928635775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a8101de0 0x7f92a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.262+0000 7f92aea5c640 1 -- 172.21.15.103:0/928635775 >> 172.21.15.103:0/928635775 conn(0x7f92a80fd880 msgr2=0x7f92a80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:09.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.262+0000 7f92aea5c640 1 -- 172.21.15.103:0/928635775 shutdown_connections 2024-09-05T02:00:09.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.262+0000 7f92aea5c640 1 -- 172.21.15.103:0/928635775 wait complete. 2024-09-05T02:00:09.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92aea5c640 1 Processor -- start 2024-09-05T02:00:09.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92aea5c640 1 -- start start 2024-09-05T02:00:09.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92aea5c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a8101de0 0x7f92a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:09.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92aea5c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8102790 0x7f92a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:09.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92aea5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a8106f10 0x7f92a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:09.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92aea5c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8111e20 con 0x7f92a8102790 2024-09-05T02:00:09.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92aea5c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8111ca0 con 0x7f92a8106f10 2024-09-05T02:00:09.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92aea5c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8111fa0 con 0x7f92a8101de0 2024-09-05T02:00:09.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92a77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8102790 0x7f92a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:09.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92a77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8102790 0x7f92a8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60934/0 (socket says 172.21.15.103:60934) 2024-09-05T02:00:09.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92a77fe640 1 -- 172.21.15.103:0/3086100329 learned_addr learned my addr 172.21.15.103:0/3086100329 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:09.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92acff9640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a8106f10 0x7f92a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:09.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92a77fe640 1 -- 172.21.15.103:0/3086100329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a8101de0 msgr2=0x7f92a80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:09.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92a77fe640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a8101de0 0x7f92a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92a77fe640 1 -- 172.21.15.103:0/3086100329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a8106f10 msgr2=0x7f92a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:09.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92a77fe640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a8106f10 0x7f92a8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.263+0000 7f92a77fe640 1 -- 172.21.15.103:0/3086100329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a81a6760 con 0x7f92a8102790 2024-09-05T02:00:09.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.264+0000 7f92a77fe640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8102790 0x7f92a8074a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f929000c920 tx=0x7f929000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:09.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.264+0000 7f92a57fa640 1 -- 172.21.15.103:0/3086100329 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290007c70 con 0x7f92a8102790 2024-09-05T02:00:09.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.264+0000 7f92a57fa640 1 -- 172.21.15.103:0/3086100329 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9290007e10 con 0x7f92a8102790 2024-09-05T02:00:09.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.264+0000 7f92a57fa640 1 -- 172.21.15.103:0/3086100329 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290018820 con 0x7f92a8102790 2024-09-05T02:00:09.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.264+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a81a6a50 con 0x7f92a8102790 2024-09-05T02:00:09.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.264+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a81a6f90 con 0x7f92a8102790 2024-09-05T02:00:09.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.265+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a81021c0 con 0x7f92a8102790 2024-09-05T02:00:09.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.266+0000 7f92a57fa640 1 -- 172.21.15.103:0/3086100329 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f92900189c0 con 0x7f92a8102790 2024-09-05T02:00:09.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.266+0000 7f92a57fa640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92940a2b80 0x7f92940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:09.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.266+0000 7f92a7fff640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92940a2b80 0x7f92940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:09.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.267+0000 7f92a57fa640 1 -- 172.21.15.103:0/3086100329 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f92900ced60 con 0x7f92a8102790 2024-09-05T02:00:09.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.267+0000 7f92a7fff640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92940a2b80 0x7f92940a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9298005fd0 tx=0x7f9298005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:09.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.271+0000 7f92a57fa640 1 -- 172.21.15.103:0/3086100329 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9290014030 con 0x7f92a8102790 2024-09-05T02:00:09.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:09 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:09.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:09 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:09.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:09 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:00:09.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:09 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:00:09.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:09 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:09.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:09 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:09.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:09 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:09.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:09 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:00:09.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:09 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:00:09.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:09 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:00:09.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.439+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f92a810a180 con 0x7f92a8102790 2024-09-05T02:00:09.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.440+0000 7f92a57fa640 1 -- 172.21.15.103:0/3086100329 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7f929009c160 con 0x7f92a8102790 2024-09-05T02:00:09.441 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:09.441 INFO:teuthology.orchestra.run.smithi103.stdout:[ 2024-09-05T02:00:09.442 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:00:09.442 INFO:teuthology.orchestra.run.smithi103.stdout: "poolnum": 1, 2024-09-05T02:00:09.442 INFO:teuthology.orchestra.run.smithi103.stdout: "poolname": ".mgr" 2024-09-05T02:00:09.442 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:00:09.442 INFO:teuthology.orchestra.run.smithi103.stdout:] 2024-09-05T02:00:09.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.442+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92940a2b80 msgr2=0x7f92940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:09.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.442+0000 7f92aea5c640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92940a2b80 0x7f92940a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9298005fd0 tx=0x7f9298005850 comp rx=0 tx=0).stop 2024-09-05T02:00:09.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8102790 msgr2=0x7f92a8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:09.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8102790 0x7f92a8074a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f929000c920 tx=0x7f929000cde0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 shutdown_connections 2024-09-05T02:00:09.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a8106f10 0x7f92a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a8102790 0x7f92a8074a30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92940a2b80 0x7f92940a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 --2- 172.21.15.103:0/3086100329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a8101de0 0x7f92a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:09.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 >> 172.21.15.103:0/3086100329 conn(0x7f92a80fd880 msgr2=0x7f92a80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:09.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 shutdown_connections 2024-09-05T02:00:09.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:09.443+0000 7f92aea5c640 1 -- 172.21.15.103:0/3086100329 wait complete. 2024-09-05T02:00:10.116 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-05T02:00:10.117 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-05T02:00:10.362 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:10.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:10 smithi139 ceph-mon[34262]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:10.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:10 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3086100329' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T02:00:10.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:10 smithi103 ceph-mon[30902]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:10.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:10 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3086100329' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T02:00:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:10 smithi167 ceph-mon[35122]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:10 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3086100329' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.869+0000 7fde3e9d5640 1 -- 172.21.15.103:0/1816340877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38101200 msgr2=0x7fde38101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.869+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/1816340877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38101200 0x7fde38101660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009900 tx=0x7fde2c02a410 comp rx=0 tx=0).stop 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.869+0000 7fde3e9d5640 1 -- 172.21.15.103:0/1816340877 shutdown_connections 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.869+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/1816340877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde38101ba0 0x7fde3810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.869+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/1816340877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38101200 0x7fde38101660 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.869+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/1816340877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde38105ad0 0x7fde38105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.869+0000 7fde3e9d5640 1 -- 172.21.15.103:0/1816340877 >> 172.21.15.103:0/1816340877 conn(0x7fde38077210 msgr2=0x7fde380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.869+0000 7fde3e9d5640 1 -- 172.21.15.103:0/1816340877 shutdown_connections 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.869+0000 7fde3e9d5640 1 -- 172.21.15.103:0/1816340877 wait complete. 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.870+0000 7fde3e9d5640 1 Processor -- start 2024-09-05T02:00:10.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.870+0000 7fde3e9d5640 1 -- start start 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.870+0000 7fde3e9d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde38101200 0x7fde38199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.870+0000 7fde3e9d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde38101ba0 0x7fde381997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.870+0000 7fde3e9d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38105ad0 0x7fde381a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.870+0000 7fde3e9d5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde38110e20 con 0x7fde38105ad0 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.870+0000 7fde3e9d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fde38110ca0 con 0x7fde38101200 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.870+0000 7fde3e9d5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde38110fa0 con 0x7fde38101ba0 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3cf72640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38105ad0 0x7fde381a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3cf72640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38105ad0 0x7fde381a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52464/0 (socket says 172.21.15.103:52464) 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3cf72640 1 -- 172.21.15.103:0/2775315452 learned_addr learned my addr 172.21.15.103:0/2775315452 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde37fff640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde38101200 0x7fde38199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde377fe640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde38101ba0 0x7fde381997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3cf72640 1 -- 172.21.15.103:0/2775315452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde38101ba0 msgr2=0x7fde381997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3cf72640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde38101ba0 0x7fde381997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3cf72640 1 -- 172.21.15.103:0/2775315452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde38101200 msgr2=0x7fde38199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3cf72640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde38101200 0x7fde38199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3cf72640 1 -- 172.21.15.103:0/2775315452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde381a4830 con 0x7fde38105ad0 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde37fff640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde38101200 0x7fde38199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3cf72640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38105ad0 0x7fde381a42f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fde3000b9e0 tx=0x7fde3000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde357fa640 1 -- 172.21.15.103:0/2775315452 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30019070 con 0x7fde38105ad0 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde357fa640 1 -- 172.21.15.103:0/2775315452 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde30014440 con 0x7fde38105ad0 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde357fa640 1 -- 172.21.15.103:0/2775315452 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300125a0 con 0x7fde38105ad0 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde381a4a00 con 0x7fde38105ad0 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.871+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde381a4f10 con 0x7fde38105ad0 2024-09-05T02:00:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.872+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde38101660 con 0x7fde38105ad0 2024-09-05T02:00:10.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.873+0000 7fde357fa640 1 -- 172.21.15.103:0/2775315452 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fde3001c070 con 0x7fde38105ad0 2024-09-05T02:00:10.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.874+0000 7fde357fa640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fde200a2c00 0x7fde200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:10.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.874+0000 7fde37fff640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fde200a2c00 0x7fde200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:10.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.874+0000 7fde357fa640 1 -- 172.21.15.103:0/2775315452 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fde300ce140 con 0x7fde38105ad0 2024-09-05T02:00:10.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.874+0000 7fde37fff640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fde200a2c00 0x7fde200a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fde28009930 tx=0x7fde28008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:10.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:10.878+0000 7fde357fa640 1 -- 172.21.15.103:0/2775315452 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde3009b540 con 0x7fde38105ad0 2024-09-05T02:00:11.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.049+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fde381a5290 con 0x7fde38105ad0 2024-09-05T02:00:11.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.334+0000 7fde357fa640 1 -- 172.21.15.103:0/2775315452 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7fde3009ae40 con 0x7fde38105ad0 2024-09-05T02:00:11.335 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' created 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.336+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fde200a2c00 msgr2=0x7fde200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.336+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fde200a2c00 0x7fde200a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fde28009930 tx=0x7fde28008040 comp rx=0 tx=0).stop 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.336+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38105ad0 msgr2=0x7fde381a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.336+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38105ad0 0x7fde381a42f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fde3000b9e0 tx=0x7fde3000bea0 comp rx=0 tx=0).stop 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.337+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 shutdown_connections 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.337+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde38105ad0 0x7fde381a42f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.337+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fde200a2c00 0x7fde200a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.337+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde38101ba0 0x7fde381997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.337+0000 7fde3e9d5640 1 --2- 172.21.15.103:0/2775315452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde38101200 0x7fde38199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.337+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 >> 172.21.15.103:0/2775315452 conn(0x7fde38077210 msgr2=0x7fde38103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.337+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 shutdown_connections 2024-09-05T02:00:11.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:11.337+0000 7fde3e9d5640 1 -- 172.21.15.103:0/2775315452 wait complete. 2024-09-05T02:00:11.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:11 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2775315452' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T02:00:11.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:11 smithi167 ceph-mon[35122]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:11.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:11 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2775315452' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T02:00:11.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:11 smithi139 ceph-mon[34262]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:11.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:11 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2775315452' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T02:00:11.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:11 smithi103 ceph-mon[30902]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:12.014 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-05T02:00:12.261 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:12 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2775315452' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T02:00:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:12 smithi167 ceph-mon[35122]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T02:00:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:12 smithi167 ceph-mon[35122]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T02:00:12.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:12 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2775315452' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T02:00:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:12 smithi139 ceph-mon[34262]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T02:00:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:12 smithi139 ceph-mon[34262]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T02:00:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:12 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2775315452' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T02:00:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:12 smithi103 ceph-mon[30902]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T02:00:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:12 smithi103 ceph-mon[30902]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T02:00:12.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.767+0000 7fb1450b3640 1 -- 172.21.15.103:0/897820329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102ef0 msgr2=0x7fb14010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:12.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.767+0000 7fb1450b3640 1 --2- 172.21.15.103:0/897820329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102ef0 0x7fb14010da40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb138009ca0 tx=0x7fb13802a950 comp rx=0 tx=0).stop 2024-09-05T02:00:12.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.767+0000 7fb1450b3640 1 -- 172.21.15.103:0/897820329 shutdown_connections 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.767+0000 7fb1450b3640 1 --2- 172.21.15.103:0/897820329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102ef0 0x7fb14010da40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.767+0000 7fb1450b3640 1 --2- 172.21.15.103:0/897820329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb140102570 0x7fb1401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.767+0000 7fb1450b3640 1 --2- 172.21.15.103:0/897820329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb140100e20 0x7fb140101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.767+0000 7fb1450b3640 1 -- 172.21.15.103:0/897820329 >> 172.21.15.103:0/897820329 conn(0x7fb1400fc7d0 msgr2=0x7fb1400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.767+0000 7fb1450b3640 1 -- 172.21.15.103:0/897820329 shutdown_connections 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.767+0000 7fb1450b3640 1 -- 172.21.15.103:0/897820329 wait complete. 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb1450b3640 1 Processor -- start 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb1450b3640 1 -- start start 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb1450b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb140100e20 0x7fb140199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb1450b3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102570 0x7fb140199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb1450b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb140102ef0 0x7fb1401a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:12.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb1450b3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb140110ed0 con 0x7fb140102570 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb1450b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb140110d50 con 0x7fb140100e20 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb1450b3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb140111050 con 0x7fb140102ef0 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb137fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102570 0x7fb140199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb137fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102570 0x7fb140199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52484/0 (socket says 172.21.15.103:52484) 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb137fff640 1 -- 172.21.15.103:0/3275129401 learned_addr learned my addr 172.21.15.103:0/3275129401 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb13ed9d640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb140100e20 0x7fb140199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb13f59e640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb140102ef0 0x7fb1401a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb137fff640 1 -- 172.21.15.103:0/3275129401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb140102ef0 msgr2=0x7fb1401a4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.768+0000 7fb137fff640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb140102ef0 0x7fb1401a4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.769+0000 7fb137fff640 1 -- 172.21.15.103:0/3275129401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb140100e20 msgr2=0x7fb140199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.769+0000 7fb137fff640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb140100e20 0x7fb140199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.769+0000 7fb137fff640 1 -- 172.21.15.103:0/3275129401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1401a49c0 con 0x7fb140102570 2024-09-05T02:00:12.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.769+0000 7fb137fff640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102570 0x7fb140199940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b4a0 tx=0x7fb12800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.769+0000 7fb13cd99640 1 -- 172.21.15.103:0/3275129401 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128015020 con 0x7fb140102570 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.769+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1401a4b90 con 0x7fb140102570 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.769+0000 7fb13cd99640 1 -- 172.21.15.103:0/3275129401 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb128011ce0 con 0x7fb140102570 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.769+0000 7fb13cd99640 1 -- 172.21.15.103:0/3275129401 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128010dc0 con 0x7fb140102570 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.769+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1401a5050 con 0x7fb140102570 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.770+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb140101280 con 0x7fb140102570 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.771+0000 7fb13cd99640 1 -- 172.21.15.103:0/3275129401 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb128018020 con 0x7fb140102570 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.772+0000 7fb13cd99640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb12c0a2b80 0x7fb12c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.772+0000 7fb13ed9d640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb12c0a2b80 0x7fb12c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:12.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.772+0000 7fb13cd99640 1 -- 172.21.15.103:0/3275129401 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7fb1280ceb80 con 0x7fb140102570 2024-09-05T02:00:12.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.772+0000 7fb13ed9d640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb12c0a2b80 0x7fb12c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb130009990 tx=0x7fb130009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:12.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.775+0000 7fb13cd99640 1 -- 172.21.15.103:0/3275129401 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb12801f020 con 0x7fb140102570 2024-09-05T02:00:12.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:12.944+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fb140109090 con 0x7fb140102570 2024-09-05T02:00:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.337+0000 7fb13cd99640 1 -- 172.21.15.103:0/3275129401 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v78) ==== 171+0+0 (secure 0 0 0) 0x7fb12809bdb0 con 0x7fb140102570 2024-09-05T02:00:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' created 2024-09-05T02:00:13.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb12c0a2b80 msgr2=0x7fb12c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:13.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb12c0a2b80 0x7fb12c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb130009990 tx=0x7fb130009110 comp rx=0 tx=0).stop 2024-09-05T02:00:13.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102570 msgr2=0x7fb140199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:13.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102570 0x7fb140199940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b4a0 tx=0x7fb12800b8a0 comp rx=0 tx=0).stop 2024-09-05T02:00:13.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 shutdown_connections 2024-09-05T02:00:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb12c0a2b80 0x7fb12c0a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb140102ef0 0x7fb1401a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb140102570 0x7fb140199940 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 --2- 172.21.15.103:0/3275129401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb140100e20 0x7fb140199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.340+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 >> 172.21.15.103:0/3275129401 conn(0x7fb1400fc7d0 msgr2=0x7fb1400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.341+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 shutdown_connections 2024-09-05T02:00:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:13.341+0000 7fb1450b3640 1 -- 172.21.15.103:0/3275129401 wait complete. 2024-09-05T02:00:13.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:13 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3275129401' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T02:00:13.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:13 smithi167 ceph-mon[35122]: pgmap v271: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:13.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:13 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3275129401' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T02:00:13.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:13 smithi167 ceph-mon[35122]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T02:00:13.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:13 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3275129401' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T02:00:13.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:13 smithi139 ceph-mon[34262]: pgmap v271: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:13.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:13 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3275129401' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T02:00:13.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:13 smithi139 ceph-mon[34262]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T02:00:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:13 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3275129401' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T02:00:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:13 smithi103 ceph-mon[30902]: pgmap v271: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:13.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:13 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3275129401' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T02:00:13.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:13 smithi103 ceph-mon[30902]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T02:00:14.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-05T02:00:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:14.360 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:14 smithi103 ceph-mon[30902]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T02:00:14.360 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:14 smithi103 ceph-mon[30902]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T02:00:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:14 smithi167 ceph-mon[35122]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T02:00:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:14 smithi167 ceph-mon[35122]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T02:00:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:14 smithi139 ceph-mon[34262]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T02:00:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:14 smithi139 ceph-mon[34262]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T02:00:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 -- 172.21.15.103:0/2688574314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8100e20 msgr2=0x7f03f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 --2- 172.21.15.103:0/2688574314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8100e20 0x7f03f8101200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009ca0 tx=0x7f03e802a950 comp rx=0 tx=0).stop 2024-09-05T02:00:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 -- 172.21.15.103:0/2688574314 shutdown_connections 2024-09-05T02:00:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 --2- 172.21.15.103:0/2688574314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f8102ef0 0x7f03f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 --2- 172.21.15.103:0/2688574314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f8102570 0x7f03f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 --2- 172.21.15.103:0/2688574314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8100e20 0x7f03f8101200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 -- 172.21.15.103:0/2688574314 >> 172.21.15.103:0/2688574314 conn(0x7f03f80fc7d0 msgr2=0x7f03f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 -- 172.21.15.103:0/2688574314 shutdown_connections 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 -- 172.21.15.103:0/2688574314 wait complete. 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.871+0000 7f03fde82640 1 Processor -- start 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03fde82640 1 -- start start 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03fde82640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f8100e20 0x7f03f81974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03fde82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8102570 0x7f03f81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03fde82640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f8102ef0 0x7f03f819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03fde82640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8110ed0 con 0x7f03f8102570 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03fde82640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8110d50 con 0x7f03f8102ef0 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03fde82640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8111050 con 0x7f03f8100e20 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8102570 0x7f03f81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8102570 0x7f03f81979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52502/0 (socket says 172.21.15.103:52502) 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f6ffd640 1 -- 172.21.15.103:0/947428137 learned_addr learned my addr 172.21.15.103:0/947428137 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f7fff640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f8102ef0 0x7f03f819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f77fe640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f8100e20 0x7f03f81974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f6ffd640 1 -- 172.21.15.103:0/947428137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f8100e20 msgr2=0x7f03f81974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f6ffd640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f8100e20 0x7f03f81974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f6ffd640 1 -- 172.21.15.103:0/947428137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f8102ef0 msgr2=0x7f03f819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f6ffd640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f8102ef0 0x7f03f819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f6ffd640 1 -- 172.21.15.103:0/947428137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f819c460 con 0x7f03f8102570 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.872+0000 7f03f7fff640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f8102ef0 0x7f03f819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.873+0000 7f03f6ffd640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8102570 0x7f03f81979f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f03e000b4a0 tx=0x7f03e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.873+0000 7f03f4ff9640 1 -- 172.21.15.103:0/947428137 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0014070 con 0x7f03f8102570 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.873+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f819c750 con 0x7f03f8102570 2024-09-05T02:00:14.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.873+0000 7f03f4ff9640 1 -- 172.21.15.103:0/947428137 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03e0010ac0 con 0x7f03f8102570 2024-09-05T02:00:14.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.873+0000 7f03f4ff9640 1 -- 172.21.15.103:0/947428137 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0017c30 con 0x7f03f8102570 2024-09-05T02:00:14.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.873+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f81a4030 con 0x7f03f8102570 2024-09-05T02:00:14.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.874+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f8101280 con 0x7f03f8102570 2024-09-05T02:00:14.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.875+0000 7f03f4ff9640 1 -- 172.21.15.103:0/947428137 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f03e0017070 con 0x7f03f8102570 2024-09-05T02:00:14.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.876+0000 7f03f4ff9640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03e40a2b80 0x7f03e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:14.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.876+0000 7f03f77fe640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03e40a2b80 0x7f03e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:14.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.876+0000 7f03f4ff9640 1 -- 172.21.15.103:0/947428137 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f03e00ce280 con 0x7f03f8102570 2024-09-05T02:00:14.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.876+0000 7f03f77fe640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03e40a2b80 0x7f03e40a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03e80093d0 tx=0x7f03e8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:14.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:14.880+0000 7f03f4ff9640 1 -- 172.21.15.103:0/947428137 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03e009b390 con 0x7f03f8102570 2024-09-05T02:00:15.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.084+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f03f8109090 con 0x7f03f8102570 2024-09-05T02:00:15.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.365+0000 7f03f4ff9640 1 -- 172.21.15.103:0/947428137 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f03e009ac90 con 0x7f03f8102570 2024-09-05T02:00:15.367 INFO:teuthology.orchestra.run.smithi103.stderr:new fs with metadata pool 2 and data pool 3 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03e40a2b80 msgr2=0x7f03e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03e40a2b80 0x7f03e40a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03e80093d0 tx=0x7f03e8009360 comp rx=0 tx=0).stop 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8102570 msgr2=0x7f03f81979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8102570 0x7f03f81979f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f03e000b4a0 tx=0x7f03e000b960 comp rx=0 tx=0).stop 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 shutdown_connections 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f8102ef0 0x7f03f819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03f8102570 0x7f03f81979f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03e40a2b80 0x7f03e40a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 --2- 172.21.15.103:0/947428137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f8100e20 0x7f03f81974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 >> 172.21.15.103:0/947428137 conn(0x7f03f80fc7d0 msgr2=0x7f03f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:15.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 shutdown_connections 2024-09-05T02:00:15.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:15.368+0000 7f03fde82640 1 -- 172.21.15.103:0/947428137 wait complete. 2024-09-05T02:00:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:15 smithi167 ceph-mon[35122]: pgmap v274: 97 pgs: 12 creating+peering, 26 active+clean, 59 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:15 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/947428137' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T02:00:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:15 smithi167 ceph-mon[35122]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T02:00:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:15 smithi139 ceph-mon[34262]: pgmap v274: 97 pgs: 12 creating+peering, 26 active+clean, 59 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:15 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/947428137' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T02:00:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:15 smithi139 ceph-mon[34262]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T02:00:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:15 smithi103 ceph-mon[30902]: pgmap v274: 97 pgs: 12 creating+peering, 26 active+clean, 59 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:15 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/947428137' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T02:00:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:15 smithi103 ceph-mon[30902]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T02:00:16.049 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-05T02:00:16.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-05T02:00:16.297 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:16 smithi167 ceph-mon[35122]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T02:00:16.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:16 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/947428137' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T02:00:16.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:16 smithi167 ceph-mon[35122]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T02:00:16.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:16 smithi167 ceph-mon[35122]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T02:00:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:16 smithi139 ceph-mon[34262]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T02:00:16.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:16 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/947428137' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T02:00:16.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:16 smithi139 ceph-mon[34262]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T02:00:16.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:16 smithi139 ceph-mon[34262]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T02:00:16.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:16 smithi103 ceph-mon[30902]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T02:00:16.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:16 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/947428137' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T02:00:16.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:16 smithi103 ceph-mon[30902]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T02:00:16.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:16 smithi103 ceph-mon[30902]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T02:00:16.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.798+0000 7f6654b61640 1 -- 172.21.15.103:0/3848835048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650100a80 msgr2=0x7f6650100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:16.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.798+0000 7f6654b61640 1 --2- 172.21.15.103:0/3848835048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650100a80 0x7f6650100ee0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6644009930 tx=0x7f664402a420 comp rx=0 tx=0).stop 2024-09-05T02:00:16.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.798+0000 7f6654b61640 1 -- 172.21.15.103:0/3848835048 shutdown_connections 2024-09-05T02:00:16.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.798+0000 7f6654b61640 1 --2- 172.21.15.103:0/3848835048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6650101420 0x7f665010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:16.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.798+0000 7f6654b61640 1 --2- 172.21.15.103:0/3848835048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650100a80 0x7f6650100ee0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.798+0000 7f6654b61640 1 --2- 172.21.15.103:0/3848835048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650106a80 0x7f6650106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.798+0000 7f6654b61640 1 -- 172.21.15.103:0/3848835048 >> 172.21.15.103:0/3848835048 conn(0x7f66500fc7a0 msgr2=0x7f66500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 -- 172.21.15.103:0/3848835048 shutdown_connections 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 -- 172.21.15.103:0/3848835048 wait complete. 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 Processor -- start 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 -- start start 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650100a80 0x7f66501993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650101420 0x7f66501998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6650106a80 0x7f66501a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6650110dd0 con 0x7f6650101420 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6650110c50 con 0x7f6650100a80 2024-09-05T02:00:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.799+0000 7f6654b61640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6650110f50 con 0x7f6650106a80 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650101420 0x7f66501998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650101420 0x7f66501998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52520/0 (socket says 172.21.15.103:52520) 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664dd9b640 1 -- 172.21.15.103:0/542798427 learned_addr learned my addr 172.21.15.103:0/542798427 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664e59c640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650100a80 0x7f66501993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664dd9b640 1 -- 172.21.15.103:0/542798427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6650106a80 msgr2=0x7f66501a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664ed9d640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6650106a80 0x7f66501a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664dd9b640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6650106a80 0x7f66501a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664dd9b640 1 -- 172.21.15.103:0/542798427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650100a80 msgr2=0x7f66501993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664dd9b640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650100a80 0x7f66501993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664dd9b640 1 -- 172.21.15.103:0/542798427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66501a4960 con 0x7f6650101420 2024-09-05T02:00:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f664dd9b640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650101420 0x7f66501998e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6644005940 tx=0x7f66440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f663f7fe640 1 -- 172.21.15.103:0/542798427 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664403f030 con 0x7f6650101420 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f663f7fe640 1 -- 172.21.15.103:0/542798427 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6644003cb0 con 0x7f6650101420 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66501a4b30 con 0x7f6650101420 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f663f7fe640 1 -- 172.21.15.103:0/542798427 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644035070 con 0x7f6650101420 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.800+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66501a4f80 con 0x7f6650101420 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.801+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6650100f60 con 0x7f6650101420 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.802+0000 7f663f7fe640 1 -- 172.21.15.103:0/542798427 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6644042030 con 0x7f6650101420 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.803+0000 7f663f7fe640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f66380a2b80 0x7f66380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.804+0000 7f664e59c640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f66380a2b80 0x7f66380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.804+0000 7f663f7fe640 1 -- 172.21.15.103:0/542798427 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f66440ef820 con 0x7f6650101420 2024-09-05T02:00:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.804+0000 7f664e59c640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f66380a2b80 0x7f66380a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6640009890 tx=0x7f6640008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:16.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.807+0000 7f663f7fe640 1 -- 172.21.15.103:0/542798427 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66440bc830 con 0x7f6650101420 2024-09-05T02:00:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:16.982+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f665006ad80 con 0x7f6650101420 2024-09-05T02:00:17.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.377+0000 7f663f7fe640 1 -- 172.21.15.103:0/542798427 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) ==== 157+0+0 (secure 0 0 0) 0x7f66440bc130 con 0x7f6650101420 2024-09-05T02:00:17.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f66380a2b80 msgr2=0x7f66380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f66380a2b80 0x7f66380a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6640009890 tx=0x7f6640008040 comp rx=0 tx=0).stop 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650101420 msgr2=0x7f66501998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650101420 0x7f66501998e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6644005940 tx=0x7f66440042f0 comp rx=0 tx=0).stop 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 shutdown_connections 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f66380a2b80 0x7f66380a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6650106a80 0x7f66501a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6650101420 0x7f66501998e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 --2- 172.21.15.103:0/542798427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650100a80 0x7f66501993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 >> 172.21.15.103:0/542798427 conn(0x7f66500fc7a0 msgr2=0x7f66500fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 shutdown_connections 2024-09-05T02:00:17.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:17.380+0000 7f6654b61640 1 -- 172.21.15.103:0/542798427 wait complete. 2024-09-05T02:00:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:17 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] up:active 2024-09-05T02:00:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:17 smithi167 ceph-mon[35122]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:17 smithi167 ceph-mon[35122]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:17 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/542798427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T02:00:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:17 smithi167 ceph-mon[35122]: pgmap v276: 97 pgs: 12 creating+peering, 53 active+clean, 32 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:17.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:17 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] up:active 2024-09-05T02:00:17.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:17 smithi139 ceph-mon[34262]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:17.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:17 smithi139 ceph-mon[34262]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:17.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:17 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/542798427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T02:00:17.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:17 smithi139 ceph-mon[34262]: pgmap v276: 97 pgs: 12 creating+peering, 53 active+clean, 32 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:17 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] up:active 2024-09-05T02:00:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:17 smithi103 ceph-mon[30902]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:17 smithi103 ceph-mon[30902]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:17 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/542798427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T02:00:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:17 smithi103 ceph-mon[30902]: pgmap v276: 97 pgs: 12 creating+peering, 53 active+clean, 32 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:18.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-05T02:00:18.302 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:18 smithi167 ceph-mon[35122]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T02:00:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:18 smithi167 ceph-mon[35122]: Cluster is now healthy 2024-09-05T02:00:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:18 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/542798427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T02:00:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:18 smithi167 ceph-mon[35122]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T02:00:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:18 smithi139 ceph-mon[34262]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T02:00:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:18 smithi139 ceph-mon[34262]: Cluster is now healthy 2024-09-05T02:00:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:18 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/542798427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T02:00:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:18 smithi139 ceph-mon[34262]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T02:00:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:18 smithi103 ceph-mon[30902]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T02:00:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:18 smithi103 ceph-mon[30902]: Cluster is now healthy 2024-09-05T02:00:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:18 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/542798427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T02:00:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:18 smithi103 ceph-mon[30902]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T02:00:18.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.778+0000 7ff4bbabd640 1 -- 172.21.15.103:0/3681543589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b41055b0 msgr2=0x7ff4b410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:18.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.778+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/3681543589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b41055b0 0x7ff4b410d130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009900 tx=0x7ff4ac02a410 comp rx=0 tx=0).stop 2024-09-05T02:00:18.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.778+0000 7ff4bbabd640 1 -- 172.21.15.103:0/3681543589 shutdown_connections 2024-09-05T02:00:18.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.778+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/3681543589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b41055b0 0x7ff4b410d130 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:18.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.778+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/3681543589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b4100e30 0x7ff4b4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.778+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/3681543589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b4100480 0x7ff4b4100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.778+0000 7ff4bbabd640 1 -- 172.21.15.103:0/3681543589 >> 172.21.15.103:0/3681543589 conn(0x7ff4b4077210 msgr2=0x7ff4b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.779+0000 7ff4bbabd640 1 -- 172.21.15.103:0/3681543589 shutdown_connections 2024-09-05T02:00:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.779+0000 7ff4bbabd640 1 -- 172.21.15.103:0/3681543589 wait complete. 2024-09-05T02:00:18.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.779+0000 7ff4bbabd640 1 Processor -- start 2024-09-05T02:00:18.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.779+0000 7ff4bbabd640 1 -- start start 2024-09-05T02:00:18.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4bbabd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b4100480 0x7ff4b4197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:18.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4bbabd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b4100e30 0x7ff4b4197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4bbabd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b41055b0 0x7ff4b419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4bbabd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4110560 con 0x7ff4b4100e30 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4bbabd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41103e0 con 0x7ff4b4100480 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4bbabd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41106e0 con 0x7ff4b41055b0 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9058640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b4100e30 0x7ff4b4197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9058640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b4100e30 0x7ff4b4197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52550/0 (socket says 172.21.15.103:52550) 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9058640 1 -- 172.21.15.103:0/1852727130 learned_addr learned my addr 172.21.15.103:0/1852727130 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9859640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b4100480 0x7ff4b4197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9058640 1 -- 172.21.15.103:0/1852727130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b41055b0 msgr2=0x7ff4b419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9058640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b41055b0 0x7ff4b419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9058640 1 -- 172.21.15.103:0/1852727130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b4100480 msgr2=0x7ff4b4197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9058640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b4100480 0x7ff4b4197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:18.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9058640 1 -- 172.21.15.103:0/1852727130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b419c3f0 con 0x7ff4b4100e30 2024-09-05T02:00:18.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4b9058640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b4100e30 0x7ff4b4197980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400b4a0 tx=0x7ff4a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:18.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4aaffd640 1 -- 172.21.15.103:0/1852727130 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4014070 con 0x7ff4b4100e30 2024-09-05T02:00:18.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4aaffd640 1 -- 172.21.15.103:0/1852727130 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4a4010ac0 con 0x7ff4b4100e30 2024-09-05T02:00:18.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4aaffd640 1 -- 172.21.15.103:0/1852727130 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4017c30 con 0x7ff4b4100e30 2024-09-05T02:00:18.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b419c6e0 con 0x7ff4b4100e30 2024-09-05T02:00:18.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.780+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b41a3fc0 con 0x7ff4b4100e30 2024-09-05T02:00:18.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.781+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b41008e0 con 0x7ff4b4100e30 2024-09-05T02:00:18.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.783+0000 7ff4aaffd640 1 -- 172.21.15.103:0/1852727130 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff4a4017070 con 0x7ff4b4100e30 2024-09-05T02:00:18.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.783+0000 7ff4aaffd640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff4a00a2c00 0x7ff4a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:18.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.783+0000 7ff4b9859640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff4a00a2c00 0x7ff4a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:18.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.783+0000 7ff4aaffd640 1 -- 172.21.15.103:0/1852727130 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8277+0+0 (secure 0 0 0) 0x7ff4a40ce470 con 0x7ff4b4100e30 2024-09-05T02:00:18.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.784+0000 7ff4b9859640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff4a00a2c00 0x7ff4a00a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009f20 tx=0x7ff4b00094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.787+0000 7ff4aaffd640 1 -- 172.21.15.103:0/1852727130 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4a409b3a0 con 0x7ff4b4100e30 2024-09-05T02:00:18.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:18.954+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff4b41a4220 con 0x7ff4b4100e30 2024-09-05T02:00:19.382 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:19 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b[65040]: 2024-09-05T02:00:19.093+0000 7f1baa8ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-05T02:00:19.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:19 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1852727130' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T02:00:19.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:19 smithi167 ceph-mon[35122]: pgmap v278: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 713 B/s wr, 5 op/s 2024-09-05T02:00:19.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:19 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1852727130' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T02:00:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:19 smithi139 ceph-mon[34262]: pgmap v278: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 713 B/s wr, 5 op/s 2024-09-05T02:00:19.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:19 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1852727130' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T02:00:19.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:19 smithi103 ceph-mon[30902]: pgmap v278: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 713 B/s wr, 5 op/s 2024-09-05T02:00:20.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.390+0000 7ff4aaffd640 1 -- 172.21.15.103:0/1852727130 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) ==== 228+0+0 (secure 0 0 0) 0x7ff4a409aca0 con 0x7ff4b4100e30 2024-09-05T02:00:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data_ec' created 2024-09-05T02:00:20.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff4a00a2c00 msgr2=0x7ff4a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:20.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff4a00a2c00 0x7ff4a00a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009f20 tx=0x7ff4b00094f0 comp rx=0 tx=0).stop 2024-09-05T02:00:20.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b4100e30 msgr2=0x7ff4b4197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:20.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b4100e30 0x7ff4b4197980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400b4a0 tx=0x7ff4a400b960 comp rx=0 tx=0).stop 2024-09-05T02:00:20.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 shutdown_connections 2024-09-05T02:00:20.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff4a00a2c00 0x7ff4a00a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:20.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b41055b0 0x7ff4b419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:20.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b4100e30 0x7ff4b4197980 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:20.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 --2- 172.21.15.103:0/1852727130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b4100480 0x7ff4b4197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:20.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 >> 172.21.15.103:0/1852727130 conn(0x7ff4b4077210 msgr2=0x7ff4b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:20.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.393+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 shutdown_connections 2024-09-05T02:00:20.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:20.394+0000 7ff4bbabd640 1 -- 172.21.15.103:0/1852727130 wait complete. 2024-09-05T02:00:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:20 smithi139 ceph-mon[34262]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T02:00:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:20 smithi103 ceph-mon[30902]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T02:00:20.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:20 smithi167 ceph-mon[35122]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T02:00:21.067 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-05T02:00:21.317 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:21.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:21 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1852727130' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T02:00:21.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:21 smithi103 ceph-mon[30902]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T02:00:21.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:21 smithi103 ceph-mon[30902]: pgmap v281: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-05T02:00:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.768+0000 7f94afdb8640 1 -- 172.21.15.103:0/1693833768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8102790 msgr2=0x7f94a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.768+0000 7f94afdb8640 1 --2- 172.21.15.103:0/1693833768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8102790 0x7f94a81067e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9498009900 tx=0x7f949802a410 comp rx=0 tx=0).stop 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 -- 172.21.15.103:0/1693833768 shutdown_connections 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 --2- 172.21.15.103:0/1693833768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94a8106f10 0x7f94a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 --2- 172.21.15.103:0/1693833768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8102790 0x7f94a81067e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 --2- 172.21.15.103:0/1693833768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a8101de0 0x7f94a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 -- 172.21.15.103:0/1693833768 >> 172.21.15.103:0/1693833768 conn(0x7f94a80fd8c0 msgr2=0x7f94a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 -- 172.21.15.103:0/1693833768 shutdown_connections 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 -- 172.21.15.103:0/1693833768 wait complete. 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 Processor -- start 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 -- start start 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94a8101de0 0x7f94a81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a8102790 0x7f94a8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8106f10 0x7f94a819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8111e20 con 0x7f94a8106f10 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8111ca0 con 0x7f94a8101de0 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94afdb8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8111fa0 con 0x7f94a8102790 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.769+0000 7f94ae355640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8106f10 0x7f94a819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94ae355640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8106f10 0x7f94a819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49672/0 (socket says 172.21.15.103:49672) 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94ae355640 1 -- 172.21.15.103:0/34792967 learned_addr learned my addr 172.21.15.103:0/34792967 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94adb54640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94a8101de0 0x7f94a81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94ad353640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a8102790 0x7f94a8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94ae355640 1 -- 172.21.15.103:0/34792967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a8102790 msgr2=0x7f94a8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94ae355640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a8102790 0x7f94a8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94ae355640 1 -- 172.21.15.103:0/34792967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94a8101de0 msgr2=0x7f94a81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94ae355640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94a8101de0 0x7f94a81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94ae355640 1 -- 172.21.15.103:0/34792967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a819d3a0 con 0x7f94a8106f10 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94ae355640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8106f10 0x7f94a819ccc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f94a000cad0 tx=0x7f94a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f949effd640 1 -- 172.21.15.103:0/34792967 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a000ae20 con 0x7f94a8106f10 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a819d690 con 0x7f94a8106f10 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f949effd640 1 -- 172.21.15.103:0/34792967 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94a0005020 con 0x7f94a8106f10 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f949effd640 1 -- 172.21.15.103:0/34792967 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a000f6b0 con 0x7f94a8106f10 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.770+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81a4f70 con 0x7f94a8106f10 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.771+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a81021c0 con 0x7f94a8106f10 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.772+0000 7f949effd640 1 -- 172.21.15.103:0/34792967 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f94a00051d0 con 0x7f94a8106f10 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.773+0000 7f949effd640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f94940a2b80 0x7f94940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.773+0000 7f94adb54640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f94940a2b80 0x7f94940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.773+0000 7f949effd640 1 -- 172.21.15.103:0/34792967 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f94a00cf250 con 0x7f94a8106f10 2024-09-05T02:00:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.773+0000 7f94adb54640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f94940a2b80 0x7f94940a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f94a4005fd0 tx=0x7f94a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.777+0000 7f949effd640 1 -- 172.21.15.103:0/34792967 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94a0014030 con 0x7f94a8106f10 2024-09-05T02:00:21.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:21 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1852727130' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T02:00:21.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:21 smithi167 ceph-mon[35122]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T02:00:21.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:21 smithi167 ceph-mon[35122]: pgmap v281: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-05T02:00:21.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:21 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1852727130' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T02:00:21.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:21 smithi139 ceph-mon[34262]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T02:00:21.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:21 smithi139 ceph-mon[34262]: pgmap v281: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-05T02:00:21.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:21.943+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f94a8109bc0 con 0x7f94a8106f10 2024-09-05T02:00:22.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.406+0000 7f949effd640 1 -- 172.21.15.103:0/34792967 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f94a0084e20 con 0x7f94a8106f10 2024-09-05T02:00:22.407 INFO:teuthology.orchestra.run.smithi103.stderr:set pool 4 allow_ec_overwrites to true 2024-09-05T02:00:22.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f94940a2b80 msgr2=0x7f94940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:22.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f94940a2b80 0x7f94940a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f94a4005fd0 tx=0x7f94a400b390 comp rx=0 tx=0).stop 2024-09-05T02:00:22.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8106f10 msgr2=0x7f94a819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:22.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8106f10 0x7f94a819ccc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f94a000cad0 tx=0x7f94a000cf90 comp rx=0 tx=0).stop 2024-09-05T02:00:22.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 shutdown_connections 2024-09-05T02:00:22.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8106f10 0x7f94a819ccc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:22.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f94940a2b80 0x7f94940a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:22.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a8102790 0x7f94a8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:22.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 --2- 172.21.15.103:0/34792967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94a8101de0 0x7f94a81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:22.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.408+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 >> 172.21.15.103:0/34792967 conn(0x7f94a80fd8c0 msgr2=0x7f94a80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:22.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.409+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 shutdown_connections 2024-09-05T02:00:22.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:22.409+0000 7f94afdb8640 1 -- 172.21.15.103:0/34792967 wait complete. 2024-09-05T02:00:22.740 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:22 smithi103 ceph-mon[30902]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T02:00:22.740 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:22 smithi103 ceph-mon[30902]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T02:00:22.740 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:22 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/34792967' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T02:00:22.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:22 smithi167 ceph-mon[35122]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T02:00:22.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:22 smithi167 ceph-mon[35122]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T02:00:22.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:22 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/34792967' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T02:00:22.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:22 smithi139 ceph-mon[34262]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T02:00:22.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:22 smithi139 ceph-mon[34262]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T02:00:22.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:22 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/34792967' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T02:00:23.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-05T02:00:23.356 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:23.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:23 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/34792967' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T02:00:23.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:23 smithi103 ceph-mon[30902]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T02:00:23.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:23 smithi103 ceph-mon[30902]: pgmap v284: 129 pgs: 2 creating+peering, 26 unknown, 101 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-09-05T02:00:23.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.874+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1455490315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28106f50 msgr2=0x7f6b2810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:23.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.874+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1455490315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28106f50 0x7f6b2810eaf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009cd0 tx=0x7f6b2002a8e0 comp rx=0 tx=0).stop 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1455490315 shutdown_connections 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1455490315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28106f50 0x7f6b2810eaf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1455490315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28100d00 0x7f6b28106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1455490315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b28100350 0x7f6b28100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1455490315 >> 172.21.15.103:0/1455490315 conn(0x7f6b28077210 msgr2=0x7f6b280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1455490315 shutdown_connections 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1455490315 wait complete. 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 Processor -- start 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 -- start start 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100350 0x7f6b28198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b28100d00 0x7f6b28198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28106f50 0x7f6b2819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111e50 con 0x7f6b28100350 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111cd0 con 0x7f6b28106f50 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.875+0000 7f6b2dac7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111fd0 con 0x7f6b28100d00 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100350 0x7f6b28198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100350 0x7f6b28198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49684/0 (socket says 172.21.15.103:49684) 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b277fe640 1 -- 172.21.15.103:0/1945866633 learned_addr learned my addr 172.21.15.103:0/1945866633 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b27fff640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28106f50 0x7f6b2819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b1ffff640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b28100d00 0x7f6b28198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b277fe640 1 -- 172.21.15.103:0/1945866633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b28100d00 msgr2=0x7f6b28198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b277fe640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b28100d00 0x7f6b28198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b277fe640 1 -- 172.21.15.103:0/1945866633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28106f50 msgr2=0x7f6b2819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b277fe640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28106f50 0x7f6b2819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b277fe640 1 -- 172.21.15.103:0/1945866633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2819d2c0 con 0x7f6b28100350 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b27fff640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28106f50 0x7f6b2819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b277fe640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100350 0x7f6b28198310 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c970 tx=0x7f6b1800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b257fa640 1 -- 172.21.15.103:0/1945866633 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18007bb0 con 0x7f6b28100350 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b257fa640 1 -- 172.21.15.103:0/1945866633 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b18007d50 con 0x7f6b28100350 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2819d5b0 con 0x7f6b28100350 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b257fa640 1 -- 172.21.15.103:0/1945866633 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800f450 con 0x7f6b28100350 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.876+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a4e90 con 0x7f6b28100350 2024-09-05T02:00:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.877+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b28100730 con 0x7f6b28100350 2024-09-05T02:00:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.878+0000 7f6b257fa640 1 -- 172.21.15.103:0/1945866633 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6b18004f20 con 0x7f6b28100350 2024-09-05T02:00:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.879+0000 7f6b257fa640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b140a2c00 0x7f6b140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.879+0000 7f6b1ffff640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b140a2c00 0x7f6b140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.879+0000 7f6b257fa640 1 -- 172.21.15.103:0/1945866633 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f6b180cf020 con 0x7f6b28100350 2024-09-05T02:00:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.880+0000 7f6b1ffff640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b140a2c00 0x7f6b140a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b280ff7f0 tx=0x7f6b10005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:23.883+0000 7f6b257fa640 1 -- 172.21.15.103:0/1945866633 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b18014030 con 0x7f6b28100350 2024-09-05T02:00:23.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:23 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/34792967' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T02:00:23.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:23 smithi167 ceph-mon[35122]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T02:00:23.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:23 smithi167 ceph-mon[35122]: pgmap v284: 129 pgs: 2 creating+peering, 26 unknown, 101 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-09-05T02:00:23.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:23 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/34792967' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T02:00:23.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:23 smithi139 ceph-mon[34262]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T02:00:23.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:23 smithi139 ceph-mon[34262]: pgmap v284: 129 pgs: 2 creating+peering, 26 unknown, 101 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-09-05T02:00:24.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.084+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f6b28199670 con 0x7f6b28100350 2024-09-05T02:00:24.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.418+0000 7f6b257fa640 1 -- 172.21.15.103:0/1945866633 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f6b1809bd60 con 0x7f6b28100350 2024-09-05T02:00:24.419 INFO:teuthology.orchestra.run.smithi103.stderr:added data pool 4 to fsmap 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b140a2c00 msgr2=0x7f6b140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b140a2c00 0x7f6b140a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b280ff7f0 tx=0x7f6b10005c50 comp rx=0 tx=0).stop 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100350 msgr2=0x7f6b28198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100350 0x7f6b28198310 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c970 tx=0x7f6b1800ce30 comp rx=0 tx=0).stop 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 shutdown_connections 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28106f50 0x7f6b2819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b140a2c00 0x7f6b140a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b28100d00 0x7f6b28198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 --2- 172.21.15.103:0/1945866633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100350 0x7f6b28198310 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 >> 172.21.15.103:0/1945866633 conn(0x7f6b28077210 msgr2=0x7f6b28105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:24.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 shutdown_connections 2024-09-05T02:00:24.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:24.420+0000 7f6b2dac7640 1 -- 172.21.15.103:0/1945866633 wait complete. 2024-09-05T02:00:24.758 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:24 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1945866633' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T02:00:24.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:24 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1945866633' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T02:00:24.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:24 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1945866633' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T02:00:25.207 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:00:25.460 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:25.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:25 smithi103 ceph-mon[30902]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T02:00:25.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:25 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1945866633' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T02:00:25.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:25 smithi103 ceph-mon[30902]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:25.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:25 smithi103 ceph-mon[30902]: pgmap v286: 129 pgs: 2 creating+peering, 7 unknown, 120 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:25.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:25 smithi167 ceph-mon[35122]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T02:00:25.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:25 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1945866633' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T02:00:25.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:25 smithi167 ceph-mon[35122]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:25.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:25 smithi167 ceph-mon[35122]: pgmap v286: 129 pgs: 2 creating+peering, 7 unknown, 120 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:25.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:25 smithi139 ceph-mon[34262]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T02:00:25.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:25 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1945866633' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T02:00:25.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:25 smithi139 ceph-mon[34262]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:25.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:25 smithi139 ceph-mon[34262]: pgmap v286: 129 pgs: 2 creating+peering, 7 unknown, 120 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:25.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.950+0000 7fa4e3918640 1 -- 172.21.15.103:0/2873224283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc101450 msgr2=0x7fa4dc105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.950+0000 7fa4e3918640 1 --2- 172.21.15.103:0/2873224283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc101450 0x7fa4dc105920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009900 tx=0x7fa4cc02a470 comp rx=0 tx=0).stop 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.950+0000 7fa4e3918640 1 -- 172.21.15.103:0/2873224283 shutdown_connections 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.950+0000 7fa4e3918640 1 --2- 172.21.15.103:0/2873224283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4dc105f40 0x7fa4dc1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.950+0000 7fa4e3918640 1 --2- 172.21.15.103:0/2873224283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc101450 0x7fa4dc105920 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.950+0000 7fa4e3918640 1 --2- 172.21.15.103:0/2873224283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4dc100b30 0x7fa4dc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.950+0000 7fa4e3918640 1 -- 172.21.15.103:0/2873224283 >> 172.21.15.103:0/2873224283 conn(0x7fa4dc0fc7a0 msgr2=0x7fa4dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.950+0000 7fa4e3918640 1 -- 172.21.15.103:0/2873224283 shutdown_connections 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.950+0000 7fa4e3918640 1 -- 172.21.15.103:0/2873224283 wait complete. 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e3918640 1 Processor -- start 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e3918640 1 -- start start 2024-09-05T02:00:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e3918640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc100b30 0x7fa4dc1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e3918640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4dc101450 0x7fa4dc1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e3918640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4dc105f40 0x7fa4dc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e3918640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc118470 con 0x7fa4dc100b30 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e3918640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1182f0 con 0x7fa4dc101450 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e3918640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1185f0 con 0x7fa4dc105f40 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e16b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc100b30 0x7fa4dc1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e16b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc100b30 0x7fa4dc1a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49702/0 (socket says 172.21.15.103:49702) 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e16b4640 1 -- 172.21.15.103:0/3330195727 learned_addr learned my addr 172.21.15.103:0/3330195727 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e0eb3640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4dc101450 0x7fa4dc1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e1eb5640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4dc105f40 0x7fa4dc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e16b4640 1 -- 172.21.15.103:0/3330195727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4dc105f40 msgr2=0x7fa4dc19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e16b4640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4dc105f40 0x7fa4dc19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:25.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e16b4640 1 -- 172.21.15.103:0/3330195727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4dc101450 msgr2=0x7fa4dc1a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e16b4640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4dc101450 0x7fa4dc1a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e16b4640 1 -- 172.21.15.103:0/3330195727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc19efa0 con 0x7fa4dc100b30 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.951+0000 7fa4e1eb5640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4dc105f40 0x7fa4dc19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.952+0000 7fa4e16b4640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc100b30 0x7fa4dc1a4680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800c890 tx=0x7fa4d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.952+0000 7fa4d27fc640 1 -- 172.21.15.103:0/3330195727 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8004220 con 0x7fa4dc100b30 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.952+0000 7fa4d27fc640 1 -- 172.21.15.103:0/3330195727 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d80043c0 con 0x7fa4dc100b30 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.952+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc19f290 con 0x7fa4dc100b30 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.952+0000 7fa4d27fc640 1 -- 172.21.15.103:0/3330195727 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8004a00 con 0x7fa4dc100b30 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.952+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc1ab4e0 con 0x7fa4dc100b30 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.953+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4dc06b070 con 0x7fa4dc100b30 2024-09-05T02:00:25.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.954+0000 7fa4d27fc640 1 -- 172.21.15.103:0/3330195727 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa4d8004ba0 con 0x7fa4dc100b30 2024-09-05T02:00:25.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.955+0000 7fa4d27fc640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4c80a2b80 0x7fa4c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:25.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.955+0000 7fa4e0eb3640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4c80a2b80 0x7fa4c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:25.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.955+0000 7fa4d27fc640 1 -- 172.21.15.103:0/3330195727 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fa4d80cf410 con 0x7fa4dc100b30 2024-09-05T02:00:25.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.955+0000 7fa4e0eb3640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4c80a2b80 0x7fa4c80a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc031440 tx=0x7fa4cc0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:25.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:25.958+0000 7fa4d27fc640 1 -- 172.21.15.103:0/3330195727 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4d8014030 con 0x7fa4dc100b30 2024-09-05T02:00:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.158+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4dc19fd30 con 0x7fa4dc100b30 2024-09-05T02:00:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.159+0000 7fa4d27fc640 1 -- 172.21.15.103:0/3330195727 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11863 (secure 0 0 0) 0x7fa4dc19fd30 con 0x7fa4dc100b30 2024-09-05T02:00:26.161 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:26.161 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"btime":"2024-09-05T02:00:24:412840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:00:24.086049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-09-05T02:00:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 17 2024-09-05T02:00:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4c80a2b80 msgr2=0x7fa4c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4c80a2b80 0x7fa4c80a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc031440 tx=0x7fa4cc0095f0 comp rx=0 tx=0).stop 2024-09-05T02:00:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc100b30 msgr2=0x7fa4dc1a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc100b30 0x7fa4dc1a4680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800c890 tx=0x7fa4d800cd50 comp rx=0 tx=0).stop 2024-09-05T02:00:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 shutdown_connections 2024-09-05T02:00:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4c80a2b80 0x7fa4c80a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:26.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4dc105f40 0x7fa4dc19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:26.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4dc101450 0x7fa4dc1a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:26.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 --2- 172.21.15.103:0/3330195727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4dc100b30 0x7fa4dc1a4680 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:26.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.161+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 >> 172.21.15.103:0/3330195727 conn(0x7fa4dc0fc7a0 msgr2=0x7fa4dc10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:26.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.162+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 shutdown_connections 2024-09-05T02:00:26.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:26.162+0000 7fa4e3918640 1 -- 172.21.15.103:0/3330195727 wait complete. 2024-09-05T02:00:26.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:26 smithi103 ceph-mon[30902]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T02:00:26.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:26 smithi103 ceph-mon[30902]: Cluster is now healthy 2024-09-05T02:00:26.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:26 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3330195727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:26.868 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:00:26.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:26 smithi167 ceph-mon[35122]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T02:00:26.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:26 smithi167 ceph-mon[35122]: Cluster is now healthy 2024-09-05T02:00:26.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:26 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3330195727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:26.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:26 smithi139 ceph-mon[34262]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T02:00:26.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:26 smithi139 ceph-mon[34262]: Cluster is now healthy 2024-09-05T02:00:26.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:26 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3330195727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:27.115 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:27.516 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:27 smithi103 ceph-mon[30902]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.595+0000 7fb650e4f640 1 -- 172.21.15.103:0/3598166578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c101420 msgr2=0x7fb64c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.595+0000 7fb650e4f640 1 --2- 172.21.15.103:0/3598166578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c101420 0x7fb64c10d950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb644009cd0 tx=0x7fb64402a8e0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.595+0000 7fb650e4f640 1 -- 172.21.15.103:0/3598166578 shutdown_connections 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.595+0000 7fb650e4f640 1 --2- 172.21.15.103:0/3598166578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c101420 0x7fb64c10d950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.595+0000 7fb650e4f640 1 --2- 172.21.15.103:0/3598166578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c100a80 0x7fb64c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.595+0000 7fb650e4f640 1 --2- 172.21.15.103:0/3598166578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb64c106a80 0x7fb64c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.595+0000 7fb650e4f640 1 -- 172.21.15.103:0/3598166578 >> 172.21.15.103:0/3598166578 conn(0x7fb64c0fc7a0 msgr2=0x7fb64c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.595+0000 7fb650e4f640 1 -- 172.21.15.103:0/3598166578 shutdown_connections 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.595+0000 7fb650e4f640 1 -- 172.21.15.103:0/3598166578 wait complete. 2024-09-05T02:00:27.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb650e4f640 1 Processor -- start 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb650e4f640 1 -- start start 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb650e4f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c100a80 0x7fb64c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb650e4f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb64c101420 0x7fb64c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb650e4f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c106a80 0x7fb64c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb650e4f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c110e20 con 0x7fb64c100a80 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb650e4f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c110ca0 con 0x7fb64c106a80 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb650e4f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c110fa0 con 0x7fb64c101420 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb64a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c100a80 0x7fb64c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb64a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c100a80 0x7fb64c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49718/0 (socket says 172.21.15.103:49718) 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb64a59c640 1 -- 172.21.15.103:0/315702266 learned_addr learned my addr 172.21.15.103:0/315702266 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb649d9b640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb64c101420 0x7fb64c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:27.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb64a59c640 1 -- 172.21.15.103:0/315702266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb64c101420 msgr2=0x7fb64c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb64a59c640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb64c101420 0x7fb64c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb64ad9d640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c106a80 0x7fb64c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb64a59c640 1 -- 172.21.15.103:0/315702266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c106a80 msgr2=0x7fb64c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb64a59c640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c106a80 0x7fb64c19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.596+0000 7fb64a59c640 1 -- 172.21.15.103:0/315702266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c19d450 con 0x7fb64c100a80 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.597+0000 7fb64a59c640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c100a80 0x7fb64c1984a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00b4a0 tx=0x7fb63c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.597+0000 7fb6437fe640 1 -- 172.21.15.103:0/315702266 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c014070 con 0x7fb64c100a80 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.597+0000 7fb6437fe640 1 -- 172.21.15.103:0/315702266 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb63c010ac0 con 0x7fb64c100a80 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.597+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c19d740 con 0x7fb64c100a80 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.597+0000 7fb6437fe640 1 -- 172.21.15.103:0/315702266 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c017c30 con 0x7fb64c100a80 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.597+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c1a5020 con 0x7fb64c100a80 2024-09-05T02:00:27.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.598+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64c100f60 con 0x7fb64c100a80 2024-09-05T02:00:27.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.599+0000 7fb6437fe640 1 -- 172.21.15.103:0/315702266 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb63c017070 con 0x7fb64c100a80 2024-09-05T02:00:27.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.600+0000 7fb6437fe640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb6380a2c00 0x7fb6380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:27.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.600+0000 7fb649d9b640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb6380a2c00 0x7fb6380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:27.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.600+0000 7fb6437fe640 1 -- 172.21.15.103:0/315702266 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fb63c0ce680 con 0x7fb64c100a80 2024-09-05T02:00:27.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.600+0000 7fb649d9b640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb6380a2c00 0x7fb6380a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb64c06b7e0 tx=0x7fb634008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:27.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.604+0000 7fb6437fe640 1 -- 172.21.15.103:0/315702266 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb63c09b3a0 con 0x7fb64c100a80 2024-09-05T02:00:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.773+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb64c1a5250 con 0x7fb64c100a80 2024-09-05T02:00:27.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.774+0000 7fb6437fe640 1 -- 172.21.15.103:0/315702266 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7fb63c09aca0 con 0x7fb64c100a80 2024-09-05T02:00:27.776 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:27.777 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:00:24.085875+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:00:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb6380a2c00 msgr2=0x7fb6380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb6380a2c00 0x7fb6380a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb64c06b7e0 tx=0x7fb634008040 comp rx=0 tx=0).stop 2024-09-05T02:00:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c100a80 msgr2=0x7fb64c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c100a80 0x7fb64c1984a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00b4a0 tx=0x7fb63c00b960 comp rx=0 tx=0).stop 2024-09-05T02:00:27.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 shutdown_connections 2024-09-05T02:00:27.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c106a80 0x7fb64c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fb6380a2c00 0x7fb6380a50c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb64c101420 0x7fb64c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 --2- 172.21.15.103:0/315702266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c100a80 0x7fb64c1984a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:27.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 >> 172.21.15.103:0/315702266 conn(0x7fb64c0fc7a0 msgr2=0x7fb64c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:27.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 shutdown_connections 2024-09-05T02:00:27.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:27.777+0000 7fb650e4f640 1 -- 172.21.15.103:0/315702266 wait complete. 2024-09-05T02:00:27.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:27 smithi167 ceph-mon[35122]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:27.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:27 smithi139 ceph-mon[34262]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:28.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:00:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:28.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:28 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/315702266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:28.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:28 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/315702266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:28.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:28 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/315702266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:29.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.256+0000 7f1f77a1f640 1 -- 172.21.15.103:0/1172387688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 msgr2=0x7f1f70100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:29.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.256+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/1172387688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 0x7f1f70100ee0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009930 tx=0x7f1f6c02a420 comp rx=0 tx=0).stop 2024-09-05T02:00:29.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.256+0000 7f1f77a1f640 1 -- 172.21.15.103:0/1172387688 shutdown_connections 2024-09-05T02:00:29.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.256+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/1172387688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f70101420 0x7f1f7010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/1172387688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 0x7f1f70100ee0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/1172387688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f70106a80 0x7f1f70106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 -- 172.21.15.103:0/1172387688 >> 172.21.15.103:0/1172387688 conn(0x7f1f700fc7a0 msgr2=0x7f1f700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 -- 172.21.15.103:0/1172387688 shutdown_connections 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 -- 172.21.15.103:0/1172387688 wait complete. 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 Processor -- start 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 -- start start 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 0x7f1f701983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f70101420 0x7f1f701988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f70106a80 0x7f1f7019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70111e50 con 0x7f1f70100a80 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70111cd0 con 0x7f1f70101420 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.257+0000 7f1f77a1f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70111fd0 con 0x7f1f70106a80 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f757bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 0x7f1f701983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:29.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f757bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 0x7f1f701983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49746/0 (socket says 172.21.15.103:49746) 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f757bb640 1 -- 172.21.15.103:0/384470321 learned_addr learned my addr 172.21.15.103:0/384470321 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f74fba640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f70101420 0x7f1f701988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f75fbc640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f70106a80 0x7f1f7019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f757bb640 1 -- 172.21.15.103:0/384470321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f70106a80 msgr2=0x7f1f7019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f757bb640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f70106a80 0x7f1f7019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f757bb640 1 -- 172.21.15.103:0/384470321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f70101420 msgr2=0x7f1f701988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f757bb640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f70101420 0x7f1f701988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f757bb640 1 -- 172.21.15.103:0/384470321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7019d360 con 0x7f1f70100a80 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f75fbc640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f70106a80 0x7f1f7019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f757bb640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 0x7f1f701983b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400d950 tx=0x7f1f6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:29.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f627fc640 1 -- 172.21.15.103:0/384470321 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64017070 con 0x7f1f70100a80 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7019d650 con 0x7f1f70100a80 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f627fc640 1 -- 172.21.15.103:0/384470321 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f64009c30 con 0x7f1f70100a80 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f701a4f30 con 0x7f1f70100a80 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.258+0000 7f1f627fc640 1 -- 172.21.15.103:0/384470321 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64020060 con 0x7f1f70100a80 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.259+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f70100ee0 con 0x7f1f70100a80 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.260+0000 7f1f627fc640 1 -- 172.21.15.103:0/384470321 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1f640109a0 con 0x7f1f70100a80 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.261+0000 7f1f627fc640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f5c0a2b80 0x7f1f5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.261+0000 7f1f74fba640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f5c0a2b80 0x7f1f5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.261+0000 7f1f627fc640 1 -- 172.21.15.103:0/384470321 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f1f640cf1f0 con 0x7f1f70100a80 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.261+0000 7f1f74fba640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f5c0a2b80 0x7f1f5c0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009900 tx=0x7f1f6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.265+0000 7f1f627fc640 1 -- 172.21.15.103:0/384470321 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f64009de0 con 0x7f1f70100a80 2024-09-05T02:00:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.437+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f70108a80 con 0x7f1f70100a80 2024-09-05T02:00:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.438+0000 7f1f627fc640 1 -- 172.21.15.103:0/384470321 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f1f6409bf10 con 0x7f1f70100a80 2024-09-05T02:00:29.440 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:29.441 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:00:24.085875+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:00:29.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f5c0a2b80 msgr2=0x7f1f5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:29.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f5c0a2b80 0x7f1f5c0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009900 tx=0x7f1f6c000bc0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 msgr2=0x7f1f701983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:29.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 0x7f1f701983b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400d950 tx=0x7f1f6400de10 comp rx=0 tx=0).stop 2024-09-05T02:00:29.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 shutdown_connections 2024-09-05T02:00:29.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f5c0a2b80 0x7f1f5c0a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f70106a80 0x7f1f7019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f70101420 0x7f1f701988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 --2- 172.21.15.103:0/384470321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f70100a80 0x7f1f701983b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:29.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 >> 172.21.15.103:0/384470321 conn(0x7f1f700fc7a0 msgr2=0x7f1f700fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:29.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 shutdown_connections 2024-09-05T02:00:29.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:29.441+0000 7f1f77a1f640 1 -- 172.21.15.103:0/384470321 wait complete. 2024-09-05T02:00:29.790 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:29 smithi103 ceph-mon[30902]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:29.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:29 smithi167 ceph-mon[35122]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:29.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:29 smithi139 ceph-mon[34262]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:30.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:00:30.485 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:30.719 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:30 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/384470321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:30.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:30 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/384470321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:30.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:30 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/384470321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:30.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.943+0000 7f8caf5aa640 1 -- 172.21.15.103:0/3330061158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100aa0 msgr2=0x7f8ca810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:30.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.943+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/3330061158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100aa0 0x7f8ca810ea20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009ca0 tx=0x7f8ca002a8c0 comp rx=0 tx=0).stop 2024-09-05T02:00:30.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.943+0000 7f8caf5aa640 1 -- 172.21.15.103:0/3330061158 shutdown_connections 2024-09-05T02:00:30.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.943+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/3330061158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100aa0 0x7f8ca810ea20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:30.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.943+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/3330061158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca8100100 0x7f8ca8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:30.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.943+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/3330061158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca8106b80 0x7f8ca8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.943+0000 7f8caf5aa640 1 -- 172.21.15.103:0/3330061158 >> 172.21.15.103:0/3330061158 conn(0x7f8ca8077230 msgr2=0x7f8ca80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.943+0000 7f8caf5aa640 1 -- 172.21.15.103:0/3330061158 shutdown_connections 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.943+0000 7f8caf5aa640 1 -- 172.21.15.103:0/3330061158 wait complete. 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8caf5aa640 1 Processor -- start 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8caf5aa640 1 -- start start 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8caf5aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100100 0x7f8ca8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8caf5aa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca8100aa0 0x7f8ca8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8caf5aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca8106b80 0x7f8ca819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8caf5aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8111f00 con 0x7f8ca8100100 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8caf5aa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8111d80 con 0x7f8ca8100aa0 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8caf5aa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8112080 con 0x7f8ca8106b80 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cad346640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100100 0x7f8ca8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cad346640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100100 0x7f8ca8198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33744/0 (socket says 172.21.15.103:33744) 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cad346640 1 -- 172.21.15.103:0/783027811 learned_addr learned my addr 172.21.15.103:0/783027811 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:30.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cadb47640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca8106b80 0x7f8ca819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cacb45640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca8100aa0 0x7f8ca8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cad346640 1 -- 172.21.15.103:0/783027811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca8106b80 msgr2=0x7f8ca819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cad346640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca8106b80 0x7f8ca819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cad346640 1 -- 172.21.15.103:0/783027811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca8100aa0 msgr2=0x7f8ca8198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cad346640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca8100aa0 0x7f8ca8198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cad346640 1 -- 172.21.15.103:0/783027811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca819d2d0 con 0x7f8ca8100100 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cadb47640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca8106b80 0x7f8ca819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.944+0000 7f8cacb45640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca8100aa0 0x7f8ca8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.945+0000 7f8cad346640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100100 0x7f8ca8198320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400c920 tx=0x7f8ca400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.945+0000 7f8c9e7fc640 1 -- 172.21.15.103:0/783027811 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4007c70 con 0x7f8ca8100100 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.945+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca819d5c0 con 0x7f8ca8100100 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.945+0000 7f8c9e7fc640 1 -- 172.21.15.103:0/783027811 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ca4007e10 con 0x7f8ca8100100 2024-09-05T02:00:30.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.945+0000 7f8c9e7fc640 1 -- 172.21.15.103:0/783027811 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4018820 con 0x7f8ca8100100 2024-09-05T02:00:30.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.945+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca81a4ea0 con 0x7f8ca8100100 2024-09-05T02:00:30.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.946+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca8100560 con 0x7f8ca8100100 2024-09-05T02:00:30.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.947+0000 7f8c9e7fc640 1 -- 172.21.15.103:0/783027811 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8ca40189c0 con 0x7f8ca8100100 2024-09-05T02:00:30.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.948+0000 7f8c9e7fc640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8c940a2b80 0x7f8c940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:30.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.948+0000 7f8cacb45640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8c940a2b80 0x7f8c940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:30.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.948+0000 7f8c9e7fc640 1 -- 172.21.15.103:0/783027811 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f8ca40cf780 con 0x7f8ca8100100 2024-09-05T02:00:30.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.948+0000 7f8cacb45640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8c940a2b80 0x7f8c940a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80ff780 tx=0x7f8c98005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:30.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:30.951+0000 7f8c9e7fc640 1 -- 172.21.15.103:0/783027811 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ca4014030 con 0x7f8ca8100100 2024-09-05T02:00:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.152+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ca8199680 con 0x7f8ca8100100 2024-09-05T02:00:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.152+0000 7f8c9e7fc640 1 -- 172.21.15.103:0/783027811 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11863 (secure 0 0 0) 0x7f8ca409c4d0 con 0x7f8ca8100100 2024-09-05T02:00:31.157 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:31.157 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"btime":"2024-09-05T02:00:24:412840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:00:24.086049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 17 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.154+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8c940a2b80 msgr2=0x7f8c940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.154+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8c940a2b80 0x7f8c940a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80ff780 tx=0x7f8c98005d30 comp rx=0 tx=0).stop 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.154+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100100 msgr2=0x7f8ca8198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.154+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100100 0x7f8ca8198320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400c920 tx=0x7f8ca400cde0 comp rx=0 tx=0).stop 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.154+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 shutdown_connections 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.154+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f8c940a2b80 0x7f8c940a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.155+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca8106b80 0x7f8ca819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.155+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca8100aa0 0x7f8ca8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.155+0000 7f8caf5aa640 1 --2- 172.21.15.103:0/783027811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ca8100100 0x7f8ca8198320 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:31.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.155+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 >> 172.21.15.103:0/783027811 conn(0x7f8ca8077230 msgr2=0x7f8ca8104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:31.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.155+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 shutdown_connections 2024-09-05T02:00:31.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:31.155+0000 7f8caf5aa640 1 -- 172.21.15.103:0/783027811 wait complete. 2024-09-05T02:00:31.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:31 smithi103 ceph-mon[30902]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:31.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:31 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/783027811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:31.875 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:00:31.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:31 smithi167 ceph-mon[35122]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:31.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:31 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/783027811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:31.895 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:31 smithi139 ceph-mon[34262]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:31.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:31 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/783027811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:00:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.598+0000 7f539ef1e640 1 -- 172.21.15.103:0/521925280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398101de0 msgr2=0x7f53981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.598+0000 7f539ef1e640 1 --2- 172.21.15.103:0/521925280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398101de0 0x7f53981021c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5388009900 tx=0x7f538802a410 comp rx=0 tx=0).stop 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 -- 172.21.15.103:0/521925280 shutdown_connections 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 --2- 172.21.15.103:0/521925280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5398106f10 0x7f539810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 --2- 172.21.15.103:0/521925280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5398102790 0x7f53981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 --2- 172.21.15.103:0/521925280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398101de0 0x7f53981021c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 -- 172.21.15.103:0/521925280 >> 172.21.15.103:0/521925280 conn(0x7f53980fd8c0 msgr2=0x7f53980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 -- 172.21.15.103:0/521925280 shutdown_connections 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 -- 172.21.15.103:0/521925280 wait complete. 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 Processor -- start 2024-09-05T02:00:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 -- start start 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5398101de0 0x7f53980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.599+0000 7f539ef1e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5398102790 0x7f5398074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539ef1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398106f10 0x7f5398074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539ef1e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5398111ec0 con 0x7f5398106f10 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539ef1e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5398111d40 con 0x7f5398101de0 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539ef1e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5398112040 con 0x7f5398102790 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539d4bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398106f10 0x7f5398074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539d4bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398106f10 0x7f5398074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33770/0 (socket says 172.21.15.103:33770) 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539d4bb640 1 -- 172.21.15.103:0/3340808180 learned_addr learned my addr 172.21.15.103:0/3340808180 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539ccba640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5398101de0 0x7f53980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f5397fff640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5398102790 0x7f5398074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539d4bb640 1 -- 172.21.15.103:0/3340808180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5398102790 msgr2=0x7f5398074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539d4bb640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5398102790 0x7f5398074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539d4bb640 1 -- 172.21.15.103:0/3340808180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5398101de0 msgr2=0x7f53980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539d4bb640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5398101de0 0x7f53980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539d4bb640 1 -- 172.21.15.103:0/3340808180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53981a6a20 con 0x7f5398106f10 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f5397fff640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5398102790 0x7f5398074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539ccba640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5398101de0 0x7f53980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539d4bb640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398106f10 0x7f5398074f70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f539000b970 tx=0x7f539000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f5395ffb640 1 -- 172.21.15.103:0/3340808180 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390019070 con 0x7f5398106f10 2024-09-05T02:00:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53981a6d10 con 0x7f5398106f10 2024-09-05T02:00:32.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f5395ffb640 1 -- 172.21.15.103:0/3340808180 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5390014440 con 0x7f5398106f10 2024-09-05T02:00:32.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.600+0000 7f5395ffb640 1 -- 172.21.15.103:0/3340808180 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53900075d0 con 0x7f5398106f10 2024-09-05T02:00:32.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.601+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53981a72a0 con 0x7f5398106f10 2024-09-05T02:00:32.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.601+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5398102240 con 0x7f5398106f10 2024-09-05T02:00:32.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.602+0000 7f5395ffb640 1 -- 172.21.15.103:0/3340808180 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f539001c070 con 0x7f5398106f10 2024-09-05T02:00:32.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.603+0000 7f5395ffb640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53800a2b80 0x7f53800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:32.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.603+0000 7f539ccba640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53800a2b80 0x7f53800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:32.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.603+0000 7f5395ffb640 1 -- 172.21.15.103:0/3340808180 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f53900ce8c0 con 0x7f5398106f10 2024-09-05T02:00:32.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.604+0000 7f539ccba640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53800a2b80 0x7f53800a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5388031440 tx=0x7f53880095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:32.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.607+0000 7f5395ffb640 1 -- 172.21.15.103:0/3340808180 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f539009b5e0 con 0x7f5398106f10 2024-09-05T02:00:32.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.772+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5398075a20 con 0x7f5398106f10 2024-09-05T02:00:32.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.774+0000 7f5395ffb640 1 -- 172.21.15.103:0/3340808180 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f539009aee0 con 0x7f5398106f10 2024-09-05T02:00:32.775 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:32.775 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:00:24.085875+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:00:32.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53800a2b80 msgr2=0x7f53800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:32.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53800a2b80 0x7f53800a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5388031440 tx=0x7f53880095f0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398106f10 msgr2=0x7f5398074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:32.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398106f10 0x7f5398074f70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f539000b970 tx=0x7f539000be30 comp rx=0 tx=0).stop 2024-09-05T02:00:32.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 shutdown_connections 2024-09-05T02:00:32.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5398106f10 0x7f5398074f70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53800a2b80 0x7f53800a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5398102790 0x7f5398074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 --2- 172.21.15.103:0/3340808180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5398101de0 0x7f53980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:32.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.776+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 >> 172.21.15.103:0/3340808180 conn(0x7f53980fd8c0 msgr2=0x7f53980fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:32.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.777+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 shutdown_connections 2024-09-05T02:00:32.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:32.777+0000 7f539ef1e640 1 -- 172.21.15.103:0/3340808180 wait complete. 2024-09-05T02:00:33.099 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:32 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3340808180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:33.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:32 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3340808180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:33.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:32 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3340808180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:33.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-05T02:00:15.086875+0000', 'modified': '2024-09-05T02:00:24.086049+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24497}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.139:6839/3082363979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 3082363979}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 3082363979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24497, 'qdb_cluster': [24497]} 2024-09-05T02:00:33.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-05T02:00:33.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T02:00:33.795 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:34.095 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:33 smithi103 ceph-mon[30902]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:34.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:33 smithi167 ceph-mon[35122]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:34.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:33 smithi139 ceph-mon[34262]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3148625686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102570 msgr2=0x7fc8681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3148625686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102570 0x7fc8681029b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009900 tx=0x7fc85c02a480 comp rx=0 tx=0).stop 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3148625686 shutdown_connections 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3148625686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc868102ef0 0x7fc86810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3148625686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102570 0x7fc8681029b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3148625686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc868100e20 0x7fc868101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3148625686 >> 172.21.15.103:0/3148625686 conn(0x7fc8680fc7d0 msgr2=0x7fc8680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3148625686 shutdown_connections 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3148625686 wait complete. 2024-09-05T02:00:34.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.322+0000 7fc86d4e3640 1 Processor -- start 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc86d4e3640 1 -- start start 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc86d4e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc868100e20 0x7fc868197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc86d4e3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc868102570 0x7fc8681978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc86d4e3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102ef0 0x7fc86819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc86d4e3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc868110e80 con 0x7fc868102ef0 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc86d4e3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc868110d00 con 0x7fc868100e20 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc86d4e3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc868111000 con 0x7fc868102570 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc8677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102ef0 0x7fc86819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc866ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc868100e20 0x7fc868197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc866ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc868100e20 0x7fc868197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:48884/0 (socket says 172.21.15.103:48884) 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc8677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102ef0 0x7fc86819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33792/0 (socket says 172.21.15.103:33792) 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc866ffd640 1 -- 172.21.15.103:0/3589799795 learned_addr learned my addr 172.21.15.103:0/3589799795 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:34.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc8667fc640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc868102570 0x7fc8681978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc8677fe640 1 -- 172.21.15.103:0/3589799795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc868102570 msgr2=0x7fc8681978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc8677fe640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc868102570 0x7fc8681978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc8677fe640 1 -- 172.21.15.103:0/3589799795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc868100e20 msgr2=0x7fc868197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc8677fe640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc868100e20 0x7fc868197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc8677fe640 1 -- 172.21.15.103:0/3589799795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86819c310 con 0x7fc868102ef0 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.323+0000 7fc866ffd640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc868100e20 0x7fc868197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.324+0000 7fc8677fe640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102ef0 0x7fc86819bc30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc86000cab0 tx=0x7fc86000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.324+0000 7fc84ffff640 1 -- 172.21.15.103:0/3589799795 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000ae20 con 0x7fc868102ef0 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.324+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3589799795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86819c600 con 0x7fc868102ef0 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.324+0000 7fc84ffff640 1 -- 172.21.15.103:0/3589799795 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc86000f900 con 0x7fc868102ef0 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.324+0000 7fc84ffff640 1 -- 172.21.15.103:0/3589799795 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860018a00 con 0x7fc868102ef0 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.324+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3589799795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc868069f70 con 0x7fc868102ef0 2024-09-05T02:00:34.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc84ffff640 1 -- 172.21.15.103:0/3589799795 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc86000fab0 con 0x7fc868102ef0 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc86f8bc740 1 Processor -- start 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc86f8bc740 1 -- start start 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c119d0 0x556c05c11db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c122f0 0x556c05c1c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 0x556c05c1efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc84ffff640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8500a2b80 0x7fc8500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c05b78090 con 0x556c05c122f0 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556c05b1fa70 con 0x556c05c1cbb0 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.326+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c05b74050 con 0x556c05c119d0 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc866ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c122f0 0x556c05c1c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc866ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c122f0 0x556c05c1c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33806/0 (socket says 172.21.15.103:33806) 2024-09-05T02:00:34.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc866ffd640 1 -- 172.21.15.103:0/67838407 learned_addr learned my addr 172.21.15.103:0/67838407 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:34.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc8667fc640 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 0x556c05c1efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc84ffff640 1 -- 172.21.15.103:0/3589799795 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fc8600cf9d0 con 0x7fc868102ef0 2024-09-05T02:00:34.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc8677fe640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8500a2b80 0x7fc8500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc8667fc640 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c119d0 0x556c05c11db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc866ffd640 1 -- 172.21.15.103:0/67838407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c119d0 msgr2=0x556c05c11db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:00:34.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc866ffd640 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c119d0 0x556c05c11db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc866ffd640 1 -- 172.21.15.103:0/67838407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 msgr2=0x556c05c1efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:00:34.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc866ffd640 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 0x556c05c1efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc866ffd640 1 -- 172.21.15.103:0/67838407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c05b2c130 con 0x556c05c122f0 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc8667fc640 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 0x556c05c1efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc866ffd640 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c122f0 0x556c05c1c670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc8580095c0 tx=0x7fc858009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138d971a1b405139 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc8677fe640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8500a2b80 0x7fc8500a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc86000a920 tx=0x7fc86000a890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc84d7fa640 1 -- 172.21.15.103:0/67838407 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858008730 con 0x556c05c122f0 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc84d7fa640 1 -- 172.21.15.103:0/67838407 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc85801d070 con 0x556c05c122f0 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.327+0000 7fc84d7fa640 1 -- 172.21.15.103:0/67838407 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858018cb0 con 0x556c05c122f0 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.328+0000 7fc86f8bc740 1 -- 172.21.15.103:0/67838407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c122f0 msgr2=0x556c05c1c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.328+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c122f0 0x556c05c1c670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc8580095c0 tx=0x7fc858009a80 comp rx=0 tx=0).stop 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.328+0000 7fc86f8bc740 1 -- 172.21.15.103:0/67838407 shutdown_connections 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.328+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 0x556c05c1efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.328+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c122f0 0x556c05c1c670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.328+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/67838407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c119d0 0x556c05c11db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.328+0000 7fc86f8bc740 1 -- 172.21.15.103:0/67838407 >> 172.21.15.103:0/67838407 conn(0x556c05c0df60 msgr2=0x556c05c0e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.328+0000 7fc86f8bc740 1 -- 172.21.15.103:0/67838407 shutdown_connections 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.328+0000 7fc86f8bc740 1 -- 172.21.15.103:0/67838407 wait complete. 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc86f8bc740 1 -- start start 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc86f8bc740 1 Processor -- start 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c119d0 0x556c05c1c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 0x556c05c19770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c1cbb0 0x556c05c19cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c05c1f780 con 0x556c05c1cbb0 2024-09-05T02:00:34.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556c05b32f20 con 0x556c05c119d0 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c05c1f900 con 0x556c05c122f0 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc86f8bc740 10 client.0 _ll_register_callbacks cb 0x556c05afc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc8667fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c1cbb0 0x556c05c19cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc8667fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c1cbb0 0x556c05c19cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33810/0 (socket says 172.21.15.103:33810) 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc8667fc640 1 -- 172.21.15.103:0/1862345406 learned_addr learned my addr 172.21.15.103:0/1862345406 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc8667fc640 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c119d0 0x556c05c1c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc866ffd640 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 0x556c05c19770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc8667fc640 1 -- 172.21.15.103:0/1862345406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 msgr2=0x556c05c19770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc8667fc640 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 0x556c05c19770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc8667fc640 1 -- 172.21.15.103:0/1862345406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c119d0 msgr2=0x556c05c1c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc8667fc640 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c119d0 0x556c05c1c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.329+0000 7fc8667fc640 1 -- 172.21.15.103:0/1862345406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c058e92b0 con 0x556c05c1cbb0 2024-09-05T02:00:34.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc8667fc640 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c1cbb0 0x556c05c19cb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc85c005ab0 tx=0x7fc85c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e3cd98d410c755 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc84aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc84aff5640 1 -- 172.21.15.103:0/1862345406 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c043070 con 0x556c05c1cbb0 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc84aff5640 1 -- 172.21.15.103:0/1862345406 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fc85c003df0 con 0x556c05c1cbb0 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc86f8bc740 1 -- 172.21.15.103:0/1862345406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc85801d070 con 0x556c05c1cbb0 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc84aff5640 1 -- 172.21.15.103:0/1862345406 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc85c046040 con 0x556c05c1cbb0 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc86f8bc740 10 client.14733 fetch_fsmap learned FSMap version 17 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc86f8bc740 1 -- 172.21.15.103:0/1862345406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556c05c2b070 con 0x556c05c1cbb0 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc84aff5640 1 -- 172.21.15.103:0/1862345406 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc85801d070 con 0x556c05c1cbb0 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc84aff5640 1 -- 172.21.15.103:0/1862345406 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fc85c03a040 con 0x556c05c1cbb0 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc86f8bc740 10 client.14733 fetch_fsmap finished waiting for FSMap version 17 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc86f8bc740 10 client.14733 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc86f8bc740 20 client.14733 populate_metadata read hostname 'smithi103' 2024-09-05T02:00:34.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556c05c2d610 0x556c05c2ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc86f8bc740 4 client.14733 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.330+0000 7fc86f8bc740 1 -- 172.21.15.103:0/1862345406 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c05b85600 con 0x556c05c2d610 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.331+0000 7fc866ffd640 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556c05c2d610 0x556c05c2ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.331+0000 7fc84aff5640 10 client.14733 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.331+0000 7fc866ffd640 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556c05c2d610 0x556c05c2ea80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6b5e16322f784e server_cookie=aeb4910e666f1a16 in_seq=0 out_seq=0 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc84aff5640 1 -- 172.21.15.103:0/1862345406 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c05b85600 con 0x556c05c2d610 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc84aff5640 10 client.14733 handle_command_reply: tid=0 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 client.14733 shutdown 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 -- 172.21.15.103:0/1862345406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c1cbb0 msgr2=0x556c05c19cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c1cbb0 0x556c05c19cb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc85c005ab0 tx=0x7fc85c02a480 comp rx=0 tx=0).stop 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 -- 172.21.15.103:0/1862345406 shutdown_connections 2024-09-05T02:00:34.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556c05c2d610 0x556c05c2ea80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c1cbb0 0x556c05c19cb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 0x556c05c19770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/1862345406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c119d0 0x556c05c1c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 -- 172.21.15.103:0/1862345406 >> 172.21.15.103:0/1862345406 conn(0x556c05c0df60 msgr2=0x556c05c13c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:34.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc84aff5640 0 client.14733 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:00:34.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 -- 172.21.15.103:0/1862345406 shutdown_connections 2024-09-05T02:00:34.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 1 -- 172.21.15.103:0/1862345406 wait complete. 2024-09-05T02:00:34.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.332+0000 7fc86f8bc740 20 client.14733 trim_cache size 0 max 16384 2024-09-05T02:00:34.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.349+0000 7fc86f8bc740 1 Processor -- start 2024-09-05T02:00:34.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.349+0000 7fc86f8bc740 1 -- start start 2024-09-05T02:00:34.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c2dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c122f0 0x556c05c19730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c1cbb0 0x556c05c19c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c05c2cdc0 con 0x556c05c119d0 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556c05c2d490 con 0x556c05c122f0 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c05c2db60 con 0x556c05c1cbb0 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc8667fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c2dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc8667fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c2dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33822/0 (socket says 172.21.15.103:33822) 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc8667fc640 1 -- 172.21.15.103:0/2062863036 learned_addr learned my addr 172.21.15.103:0/2062863036 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc866ffd640 1 --2- 172.21.15.103:0/2062863036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c122f0 0x556c05c19730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc8667fc640 1 -- 172.21.15.103:0/2062863036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c1cbb0 msgr2=0x556c05c19c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc8667fc640 1 --2- 172.21.15.103:0/2062863036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c1cbb0 0x556c05c19c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc8667fc640 1 -- 172.21.15.103:0/2062863036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c122f0 msgr2=0x556c05c19730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc8667fc640 1 --2- 172.21.15.103:0/2062863036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c122f0 0x556c05c19730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc8667fc640 1 -- 172.21.15.103:0/2062863036 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c05c11800 con 0x556c05c119d0 2024-09-05T02:00:34.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc8667fc640 1 --2- 172.21.15.103:0/2062863036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c2dce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc85c052490 tx=0x7fc85c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf035cfe8fbbcb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:34.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc84cff9640 1 -- 172.21.15.103:0/2062863036 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c043070 con 0x556c05c119d0 2024-09-05T02:00:34.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.350+0000 7fc84cff9640 1 -- 172.21.15.103:0/2062863036 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc85c009520 con 0x556c05c119d0 2024-09-05T02:00:34.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc84cff9640 1 -- 172.21.15.103:0/2062863036 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c035040 con 0x556c05c119d0 2024-09-05T02:00:34.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc86f8bc740 1 -- 172.21.15.103:0/2062863036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 msgr2=0x556c05c2dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/2062863036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c2dce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc85c052490 tx=0x7fc85c0058b0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc86f8bc740 1 -- 172.21.15.103:0/2062863036 shutdown_connections 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/2062863036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c1cbb0 0x556c05c19c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/2062863036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c122f0 0x556c05c19730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/2062863036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c2dce0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc86f8bc740 1 -- 172.21.15.103:0/2062863036 >> 172.21.15.103:0/2062863036 conn(0x556c05c0df60 msgr2=0x556c05c18cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc86f8bc740 1 -- 172.21.15.103:0/2062863036 shutdown_connections 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.351+0000 7fc86f8bc740 1 -- 172.21.15.103:0/2062863036 wait complete. 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc86f8bc740 1 -- start start 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc86f8bc740 1 Processor -- start 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c31ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 0x556c05c320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc86f8bc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 0x556c05c32620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c05c2e900 con 0x556c05c119d0 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556c05c2e230 con 0x556c05c1cbb0 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc86f8bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c05c2efd0 con 0x556c05c122f0 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc8667fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c31ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc86f8bc740 10 client.0 _ll_register_callbacks cb 0x7fc85c004300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc8667fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c31ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33830/0 (socket says 172.21.15.103:33830) 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc8667fc640 1 -- 172.21.15.103:0/3638242030 learned_addr learned my addr 172.21.15.103:0/3638242030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc8667fc640 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 0x556c05c32620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.352+0000 7fc866ffd640 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 0x556c05c320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc8667fc640 1 -- 172.21.15.103:0/3638242030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 msgr2=0x556c05c320e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc8667fc640 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 0x556c05c320e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc8667fc640 1 -- 172.21.15.103:0/3638242030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 msgr2=0x556c05c32620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc8667fc640 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 0x556c05c32620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc8667fc640 1 -- 172.21.15.103:0/3638242030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c05c95d60 con 0x556c05c119d0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc8667fc640 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c31ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc85c04f040 tx=0x7fc85c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a275f6835340feb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc84aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc84aff5640 1 -- 172.21.15.103:0/3638242030 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c043070 con 0x556c05c119d0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc84aff5640 1 -- 172.21.15.103:0/3638242030 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fc85c005510 con 0x556c05c119d0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc86f8bc740 1 -- 172.21.15.103:0/3638242030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc85c009520 con 0x556c05c119d0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc84aff5640 1 -- 172.21.15.103:0/3638242030 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc85c035040 con 0x556c05c119d0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc86f8bc740 10 client.14739 fetch_fsmap learned FSMap version 17 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc86f8bc740 1 -- 172.21.15.103:0/3638242030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556c05c97060 con 0x556c05c119d0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.353+0000 7fc84aff5640 1 -- 172.21.15.103:0/3638242030 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc85c009520 con 0x556c05c119d0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc84aff5640 1 -- 172.21.15.103:0/3638242030 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fc85c03db40 con 0x556c05c119d0 2024-09-05T02:00:34.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc86f8bc740 10 client.14739 fetch_fsmap finished waiting for FSMap version 17 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc86f8bc740 10 client.14739 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc86f8bc740 20 client.14739 populate_metadata read hostname 'smithi103' 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556c05c9b7c0 0x556c05c9a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc86f8bc740 4 client.14739 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc86f8bc740 1 -- 172.21.15.103:0/3638242030 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "status"}) -- 0x556c05b7fb10 con 0x556c05c9b7c0 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc866ffd640 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556c05c9b7c0 0x556c05c9a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc84aff5640 10 client.14739 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.354+0000 7fc866ffd640 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556c05c9b7c0 0x556c05c9a4d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e867445985d74ee5 server_cookie=90c3058b24381b36 in_seq=0 out_seq=0 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc84aff5640 1 -- 172.21.15.103:0/3638242030 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556c05b7fb10 con 0x556c05c9b7c0 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc84aff5640 10 client.14739 handle_command_reply: tid=0 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 client.14739 shutdown 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 -- 172.21.15.103:0/3638242030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 msgr2=0x556c05c31ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:34.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c31ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc85c04f040 tx=0x7fc85c02a480 comp rx=0 tx=0).stop 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 -- 172.21.15.103:0/3638242030 shutdown_connections 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556c05c9b7c0 0x556c05c9a4d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc84aff5640 0 client.14739 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c05c1cbb0 0x556c05c32620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c05c122f0 0x556c05c320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 --2- 172.21.15.103:0/3638242030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c05c119d0 0x556c05c31ba0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 -- 172.21.15.103:0/3638242030 >> 172.21.15.103:0/3638242030 conn(0x556c05c0df60 msgr2=0x556c05c19730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 -- 172.21.15.103:0/3638242030 shutdown_connections 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 1 -- 172.21.15.103:0/3638242030 wait complete. 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.355+0000 7fc86f8bc740 20 client.14739 trim_cache size 0 max 16384 2024-09-05T02:00:34.361 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24497, 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 18.986431363000001, 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 17, 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 30.672047674000002 2024-09-05T02:00:34.362 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:00:34.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8500a2b80 msgr2=0x7fc8500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:34.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8500a2b80 0x7fc8500a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc86000a920 tx=0x7fc86000a890 comp rx=0 tx=0).stop 2024-09-05T02:00:34.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102ef0 msgr2=0x7fc86819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:34.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102ef0 0x7fc86819bc30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc86000cab0 tx=0x7fc86000cf70 comp rx=0 tx=0).stop 2024-09-05T02:00:34.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3589799795 shutdown_connections 2024-09-05T02:00:34.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc868102ef0 0x7fc86819bc30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8500a2b80 0x7fc8500a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc868102570 0x7fc8681978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 --2- 172.21.15.103:0/3589799795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc868100e20 0x7fc868197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:34.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3589799795 >> 172.21.15.103:0/3589799795 conn(0x7fc8680fc7d0 msgr2=0x7fc8680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:34.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3589799795 shutdown_connections 2024-09-05T02:00:34.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:34.356+0000 7fc86d4e3640 1 -- 172.21.15.103:0/3589799795 wait complete. 2024-09-05T02:00:35.047 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T02:00:35.047 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T02:00:35.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T02:00:35.296 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:35.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.812+0000 7fa168d49640 1 -- 172.21.15.103:0/987805397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164106a80 msgr2=0x7fa164106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.812+0000 7fa168d49640 1 --2- 172.21.15.103:0/987805397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164106a80 0x7fa164106e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009cd0 tx=0x7fa15c02a8e0 comp rx=0 tx=0).stop 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.813+0000 7fa168d49640 1 -- 172.21.15.103:0/987805397 shutdown_connections 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.813+0000 7fa168d49640 1 --2- 172.21.15.103:0/987805397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa164101420 0x7fa16410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.813+0000 7fa168d49640 1 --2- 172.21.15.103:0/987805397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa164100a80 0x7fa164100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.813+0000 7fa168d49640 1 --2- 172.21.15.103:0/987805397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164106a80 0x7fa164106e60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.813+0000 7fa168d49640 1 -- 172.21.15.103:0/987805397 >> 172.21.15.103:0/987805397 conn(0x7fa1640fc7a0 msgr2=0x7fa1640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.813+0000 7fa168d49640 1 -- 172.21.15.103:0/987805397 shutdown_connections 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.813+0000 7fa168d49640 1 -- 172.21.15.103:0/987805397 wait complete. 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.813+0000 7fa168d49640 1 Processor -- start 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.813+0000 7fa168d49640 1 -- start start 2024-09-05T02:00:35.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa168d49640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa164100a80 0x7fa1641983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa168d49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164101420 0x7fa1641988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa168d49640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa164106a80 0x7fa16419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa168d49640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa164110cf0 con 0x7fa164101420 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa168d49640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa164110b70 con 0x7fa164106a80 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa168d49640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa164110e70 con 0x7fa164100a80 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa161d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164101420 0x7fa1641988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa161d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164101420 0x7fa1641988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33848/0 (socket says 172.21.15.103:33848) 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa161d9b640 1 -- 172.21.15.103:0/3768763083 learned_addr learned my addr 172.21.15.103:0/3768763083 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa16259c640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa164100a80 0x7fa1641983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa162d9d640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa164106a80 0x7fa16419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa161d9b640 1 -- 172.21.15.103:0/3768763083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa164100a80 msgr2=0x7fa1641983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa161d9b640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa164100a80 0x7fa1641983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:35.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa161d9b640 1 -- 172.21.15.103:0/3768763083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa164106a80 msgr2=0x7fa16419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa161d9b640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa164106a80 0x7fa16419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa161d9b640 1 -- 172.21.15.103:0/3768763083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16419d350 con 0x7fa164101420 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa161d9b640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164101420 0x7fa1641988e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00d950 tx=0x7fa14c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa15b7fe640 1 -- 172.21.15.103:0/3768763083 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c017070 con 0x7fa164101420 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16419d640 con 0x7fa164101420 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa15b7fe640 1 -- 172.21.15.103:0/3768763083 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa14c009c30 con 0x7fa164101420 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa15b7fe640 1 -- 172.21.15.103:0/3768763083 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00f620 con 0x7fa164101420 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.814+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1641a4f20 con 0x7fa164101420 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.815+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa164100ee0 con 0x7fa164101420 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.816+0000 7fa15b7fe640 1 -- 172.21.15.103:0/3768763083 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa14c0109a0 con 0x7fa164101420 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.817+0000 7fa15b7fe640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1500a2b80 0x7fa1500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.817+0000 7fa16259c640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1500a2b80 0x7fa1500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:35.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.817+0000 7fa15b7fe640 1 -- 172.21.15.103:0/3768763083 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fa14c0cee30 con 0x7fa164101420 2024-09-05T02:00:35.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.818+0000 7fa16259c640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1500a2b80 0x7fa1500a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa15c006010 tx=0x7fa15c0099c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:35.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:35.821+0000 7fa15b7fe640 1 -- 172.21.15.103:0/3768763083 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa14c009de0 con 0x7fa164101420 2024-09-05T02:00:36.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.034+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa16410a030 con 0x7fa164101420 2024-09-05T02:00:36.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.034+0000 7fa15b7fe640 1 -- 172.21.15.103:0/3768763083 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fa14c09b500 con 0x7fa164101420 2024-09-05T02:00:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T02:00:36.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1500a2b80 msgr2=0x7fa1500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:36.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1500a2b80 0x7fa1500a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa15c006010 tx=0x7fa15c0099c0 comp rx=0 tx=0).stop 2024-09-05T02:00:36.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164101420 msgr2=0x7fa1641988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:36.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164101420 0x7fa1641988e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00d950 tx=0x7fa14c00de10 comp rx=0 tx=0).stop 2024-09-05T02:00:36.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 shutdown_connections 2024-09-05T02:00:36.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa164106a80 0x7fa16419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:36.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa164101420 0x7fa1641988e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:36.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1500a2b80 0x7fa1500a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:36.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 --2- 172.21.15.103:0/3768763083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa164100a80 0x7fa1641983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:36.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.036+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 >> 172.21.15.103:0/3768763083 conn(0x7fa1640fc7a0 msgr2=0x7fa16406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:36.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.037+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 shutdown_connections 2024-09-05T02:00:36.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36.037+0000 7fa168d49640 1 -- 172.21.15.103:0/3768763083 wait complete. 2024-09-05T02:00:36.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:36 smithi103 ceph-mon[30902]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:36.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:36 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3768763083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T02:00:36.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:36 smithi139 ceph-mon[34262]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:36.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:36 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3768763083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T02:00:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:36 smithi167 ceph-mon[35122]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:36 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3768763083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T02:00:36.800 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-05T02:00:36.800 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T02:00:36.800 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-09-05T02:00:36.800 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-05T02:00:36.800 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T02:00:36.800 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-05T02:00:36.800 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T02:00:36.801 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T02:00:36.801 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T02:00:36.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:36.801 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-09-05T02:00:36.817 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T02:00:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T02:00:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T02:00:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-05T02:00:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-09-05T02:00:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-05T02:00:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-05T02:00:36.819 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T02:00:36.819 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:00:36.819 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:00:36.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2024-09-05T02:00:36.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2024-09-05T02:00:36.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2024-09-05T02:00:36.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T02:00:36.820 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:00:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:00:37.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:00:37.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:37.020 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T02:00:37.046 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-09-05T02:00:37.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:37.047 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-09-05T02:00:37.123 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-09-05T02:00:37.123 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T02:00:37.123 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-05T02:00:37.124 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-05T02:00:37.124 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T02:00:37.124 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:00:37.124 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:00:37.124 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-05T02:00:37.124 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-05T02:00:37.124 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-05T02:00:37.124 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:00:37.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:00:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:00:37.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:37.397 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-05T02:00:37.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:37.414 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-09-05T02:00:37.470 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:00:37.470 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:00:37.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T02:00:37.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-05T02:00:37.470 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:00:37.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:00:37.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:00:37.588 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-05T02:00:37.588 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:00:37.588 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:00:37.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-05T02:00:37.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T02:00:37.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-05T02:00:37.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-05T02:00:37.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-05T02:00:37.589 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:00:37.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:00:37.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:00:37.712 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:00:37.712 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:00:37.712 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2024-09-05T02:00:37.712 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T02:00:37.712 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:00:37.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:00:37.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:00:37.790 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-05T02:00:37.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T02:00:37.790 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-05T02:00:37.805 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T02:00:37.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T02:00:37.805 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-05T02:00:37.860 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-09-05T02:00:37.926 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-09-05T02:00:37.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T02:00:37.984 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/77f6d94187da0151c1e70b3458f4b0b677f2d2efb39c4b1225614329c7e42f22/merged 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/7fbfcaa433d3550b597826fd248ccdbb900c0b6a9a167e1d7499a35ee0ffb7d4/merged 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/78a0d766eb04ba5d7d34d49cfc7468281c826fe611704a9cc0f63e424bdf7cb7/merged 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a3796d400e19b2ed58ecb15b7e9b06fe81192e20c2f3a0e084c368230f293d55/merged 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/4de6c1d701ae89ad985fab0fe831f81b6dc5b38046b7f57378a883a517ef3852/merged 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/445a00a99baf26b71312251505aeb4d570d932851a06271e2c149a9151c87b95/merged 2024-09-05T02:00:37.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/6179c953998a5ecbb9e302c4e8c2f00517884e41fa1bd24eee12d97bc9f5489a/merged 2024-09-05T02:00:37.986 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a001003b52f9e4db8bc9e1a5c58af8996a4de4eb718a7d35e6bac347b3cd0a96/merged 2024-09-05T02:00:37.986 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/444a0f4c2ba2b20895a50ac0c1043ab2b6ee41bd33a99c8ef5f9d36fbf635330/merged 2024-09-05T02:00:37.986 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/1bb8707282af87813355c77a1093e16db2b83b4475f563a3d35418727c919ad3/merged 2024-09-05T02:00:37.986 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-09-05T02:00:37.986 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T02:00:37.986 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T02:00:37.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:37.986 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-09-05T02:00:38.038 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-05T02:00:38.039 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-05T02:00:38.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-09-05T02:00:38.107 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T02:00:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-09-05T02:00:38.163 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T02:00:38.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-09-05T02:00:38.165 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/77f6d94187da0151c1e70b3458f4b0b677f2d2efb39c4b1225614329c7e42f22/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/7fbfcaa433d3550b597826fd248ccdbb900c0b6a9a167e1d7499a35ee0ffb7d4/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/78a0d766eb04ba5d7d34d49cfc7468281c826fe611704a9cc0f63e424bdf7cb7/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a3796d400e19b2ed58ecb15b7e9b06fe81192e20c2f3a0e084c368230f293d55/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/4de6c1d701ae89ad985fab0fe831f81b6dc5b38046b7f57378a883a517ef3852/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/445a00a99baf26b71312251505aeb4d570d932851a06271e2c149a9151c87b95/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/6179c953998a5ecbb9e302c4e8c2f00517884e41fa1bd24eee12d97bc9f5489a/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a001003b52f9e4db8bc9e1a5c58af8996a4de4eb718a7d35e6bac347b3cd0a96/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/444a0f4c2ba2b20895a50ac0c1043ab2b6ee41bd33a99c8ef5f9d36fbf635330/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/1bb8707282af87813355c77a1093e16db2b83b4475f563a3d35418727c919ad3/merged 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-09-05T02:00:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T02:00:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T02:00:38.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:38.167 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-09-05T02:00:38.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi103.stderr:2024-09-05T02:00:38.185+0000 7f96bd882480 -1 init, newargv = 0x5608e7ef9aa0 newargc=15 2024-09-05T02:00:38.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi103.stderr:ceph-fuse[72301]: starting ceph client 2024-09-05T02:00:38.192 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi103.stderr:ceph-fuse[72301]: starting fuse 2024-09-05T02:00:38.201 INFO:teuthology.orchestra.run.smithi103.stdout:109 2024-09-05T02:00:38.201 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [109] 2024-09-05T02:00:38.201 DEBUG:teuthology.orchestra.run.smithi103:> sudo stdin-killer -- python3 -c ' 2024-09-05T02:00:38.201 DEBUG:teuthology.orchestra.run.smithi103:> import glob 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> import re 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> import os 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> import subprocess 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> def _find_admin_socket(client_name): 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> files = glob.glob(asok_path) 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> # Given a non-glob path, it better be there 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> if "*" not in asok_path: 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> assert(len(files) == 1) 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> return files[0] 2024-09-05T02:00:38.202 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> for f in files: 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> contents = proc_f.read() 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> if mountpoint in contents: 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> return f 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> print(_find_admin_socket("client.admin")) 2024-09-05T02:00:38.203 DEBUG:teuthology.orchestra.run.smithi103:> ' 2024-09-05T02:00:38.262 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:38 smithi103 ceph-mon[30902]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:38.318 INFO:teuthology.orchestra.run.smithi103.stdout:/var/run/ceph/ceph-client.admin.72301.asok 2024-09-05T02:00:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:00:38.329 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72301.asok 2024-09-05T02:00:38.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:38.329 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72301.asok status 2024-09-05T02:00:38.395 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:38 smithi139 ceph-mon[34262]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:38.462 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:00:38.462 INFO:teuthology.orchestra.run.smithi103.stdout: "metadata": { 2024-09-05T02:00:38.462 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_sha1": "91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4", 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "entity_id": "admin", 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": "smithi103", 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "pid": "72301", 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "root": "/" 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_count": 0, 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_pinned_count": 0, 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 14751, 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "inst": { 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": { 2024-09-05T02:00:38.463 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client", 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: "num": 14751 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2062550630 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2062550630 2024-09-05T02:00:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:00:38.465 INFO:teuthology.orchestra.run.smithi103.stdout: "inst_str": "client.14751 192.168.0.1:0/2062550630", 2024-09-05T02:00:38.465 INFO:teuthology.orchestra.run.smithi103.stdout: "addr_str": "192.168.0.1:0/2062550630", 2024-09-05T02:00:38.465 INFO:teuthology.orchestra.run.smithi103.stdout: "inode_count": 1, 2024-09-05T02:00:38.465 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_epoch": 17, 2024-09-05T02:00:38.465 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch": 86, 2024-09-05T02:00:38.465 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch_barrier": 0, 2024-09-05T02:00:38.465 INFO:teuthology.orchestra.run.smithi103.stdout: "blocklisted": false, 2024-09-05T02:00:38.465 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs" 2024-09-05T02:00:38.465 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:00:38.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:38.471 DEBUG:teuthology.orchestra.run.smithi103:> stat --file-system '--printf=%T 2024-09-05T02:00:38.471 DEBUG:teuthology.orchestra.run.smithi103:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-05T02:00:38.487 INFO:teuthology.orchestra.run.smithi103.stdout:fuseblk 2024-09-05T02:00:38.487 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-05T02:00:38.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:38.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-05T02:00:38.553 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-05T02:00:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:38 smithi167 ceph-mon[35122]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:00:38.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T02:00:38.672 INFO:teuthology.orchestra.run.smithi103.stdout:# file: . 2024-09-05T02:00:38.673 INFO:teuthology.orchestra.run.smithi103.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-05T02:00:38.673 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:38.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:00:38.679 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T02:00:38.679 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T02:00:38.696 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T02:00:38.982 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:39.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 -- 172.21.15.103:0/3344790210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c101df0 msgr2=0x7fc78c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:39.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 --2- 172.21.15.103:0/3344790210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c101df0 0x7fc78c1021d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009900 tx=0x7fc77c02a410 comp rx=0 tx=0).stop 2024-09-05T02:00:39.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 -- 172.21.15.103:0/3344790210 shutdown_connections 2024-09-05T02:00:39.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 --2- 172.21.15.103:0/3344790210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc78c106f20 0x7fc78c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:39.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 --2- 172.21.15.103:0/3344790210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78c1027a0 0x7fc78c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:39.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 --2- 172.21.15.103:0/3344790210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c101df0 0x7fc78c1021d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:39.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 -- 172.21.15.103:0/3344790210 >> 172.21.15.103:0/3344790210 conn(0x7fc78c0fd8f0 msgr2=0x7fc78c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:39.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 -- 172.21.15.103:0/3344790210 shutdown_connections 2024-09-05T02:00:39.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 -- 172.21.15.103:0/3344790210 wait complete. 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 Processor -- start 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.488+0000 7fc792d45640 1 -- start start 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc792d45640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc78c101df0 0x7fc78c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc792d45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78c1027a0 0x7fc78c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc792d45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c106f20 0x7fc78c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc792d45640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c111e30 con 0x7fc78c106f20 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc792d45640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c111cb0 con 0x7fc78c101df0 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc792d45640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c111fb0 con 0x7fc78c1027a0 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc7912e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c106f20 0x7fc78c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc7912e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c106f20 0x7fc78c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55584/0 (socket says 172.21.15.103:55584) 2024-09-05T02:00:39.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc7912e2640 1 -- 172.21.15.103:0/3797204042 learned_addr learned my addr 172.21.15.103:0/3797204042 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc790ae1640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc78c101df0 0x7fc78c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc78bfff640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78c1027a0 0x7fc78c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc7912e2640 1 -- 172.21.15.103:0/3797204042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78c1027a0 msgr2=0x7fc78c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc7912e2640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78c1027a0 0x7fc78c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc7912e2640 1 -- 172.21.15.103:0/3797204042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc78c101df0 msgr2=0x7fc78c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc7912e2640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc78c101df0 0x7fc78c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc7912e2640 1 -- 172.21.15.103:0/3797204042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c19d320 con 0x7fc78c106f20 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc7912e2640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c106f20 0x7fc78c19cc40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc78400cce0 tx=0x7fc78400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc789ffb640 1 -- 172.21.15.103:0/3797204042 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78400f450 con 0x7fc78c106f20 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc789ffb640 1 -- 172.21.15.103:0/3797204042 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc78400faa0 con 0x7fc78c106f20 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c19d610 con 0x7fc78c106f20 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.489+0000 7fc789ffb640 1 -- 172.21.15.103:0/3797204042 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784018b70 con 0x7fc78c106f20 2024-09-05T02:00:39.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.490+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c1a4ef0 con 0x7fc78c106f20 2024-09-05T02:00:39.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.491+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78c1021d0 con 0x7fc78c106f20 2024-09-05T02:00:39.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.492+0000 7fc789ffb640 1 -- 172.21.15.103:0/3797204042 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc784018d10 con 0x7fc78c106f20 2024-09-05T02:00:39.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.492+0000 7fc789ffb640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc7740a2c00 0x7fc7740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:39.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.493+0000 7fc789ffb640 1 -- 172.21.15.103:0/3797204042 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fc7840cf690 con 0x7fc78c106f20 2024-09-05T02:00:39.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.493+0000 7fc790ae1640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc7740a2c00 0x7fc7740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:39.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.493+0000 7fc790ae1640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc7740a2c00 0x7fc7740a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc77c031440 tx=0x7fc77c005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:39.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.496+0000 7fc789ffb640 1 -- 172.21.15.103:0/3797204042 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc784014030 con 0x7fc78c106f20 2024-09-05T02:00:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.676+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc78c1996d0 con 0x7fc78c106f20 2024-09-05T02:00:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.676+0000 7fc789ffb640 1 -- 172.21.15.103:0/3797204042 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+601 (secure 0 0 0) 0x7fc78c1996d0 con 0x7fc78c106f20 2024-09-05T02:00:39.677 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2384335573 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:39.677 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2384335573 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:39.677 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2155492495 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:39.677 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/396485172 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:39.677 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1467689006 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:39.678 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3874104813 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:39.678 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1573405851 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:39.678 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2132077607 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:39.678 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3892241075 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:39.678 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2132077607 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-09-05T02:00:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.678+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc7740a2c00 msgr2=0x7fc7740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.678+0000 7fc792d45640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc7740a2c00 0x7fc7740a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc77c031440 tx=0x7fc77c005e80 comp rx=0 tx=0).stop 2024-09-05T02:00:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.678+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c106f20 msgr2=0x7fc78c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.678+0000 7fc792d45640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c106f20 0x7fc78c19cc40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc78400cce0 tx=0x7fc78400a620 comp rx=0 tx=0).stop 2024-09-05T02:00:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.679+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 shutdown_connections 2024-09-05T02:00:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.679+0000 7fc792d45640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc78c106f20 0x7fc78c19cc40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.679+0000 7fc792d45640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc7740a2c00 0x7fc7740a50c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.679+0000 7fc792d45640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78c1027a0 0x7fc78c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.679+0000 7fc792d45640 1 --2- 172.21.15.103:0/3797204042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc78c101df0 0x7fc78c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.679+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 >> 172.21.15.103:0/3797204042 conn(0x7fc78c0fd8f0 msgr2=0x7fc78c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.679+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 shutdown_connections 2024-09-05T02:00:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:39.679+0000 7fc792d45640 1 -- 172.21.15.103:0/3797204042 wait complete. 2024-09-05T02:00:40.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:40 smithi103 ceph-mon[30902]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T02:00:40.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:40 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3797204042' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:00:40.392 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T02:00:40.392 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T02:00:40.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:40 smithi139 ceph-mon[34262]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T02:00:40.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:40 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3797204042' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:00:40.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T02:00:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:40 smithi167 ceph-mon[35122]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T02:00:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:40 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3797204042' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:00:40.698 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:41.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.181+0000 7f974a751640 1 -- 172.21.15.103:0/1121551275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102ef0 msgr2=0x7f974410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:41.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.181+0000 7f974a751640 1 --2- 172.21.15.103:0/1121551275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102ef0 0x7f974410da40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f973c009ca0 tx=0x7f973c02a950 comp rx=0 tx=0).stop 2024-09-05T02:00:41.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.181+0000 7f974a751640 1 -- 172.21.15.103:0/1121551275 shutdown_connections 2024-09-05T02:00:41.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.181+0000 7f974a751640 1 --2- 172.21.15.103:0/1121551275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102ef0 0x7f974410da40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.181+0000 7f974a751640 1 --2- 172.21.15.103:0/1121551275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9744102570 0x7f97441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.181+0000 7f974a751640 1 --2- 172.21.15.103:0/1121551275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744100e20 0x7f9744101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.181+0000 7f974a751640 1 -- 172.21.15.103:0/1121551275 >> 172.21.15.103:0/1121551275 conn(0x7f97440fc7d0 msgr2=0x7f97440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:41.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.181+0000 7f974a751640 1 -- 172.21.15.103:0/1121551275 shutdown_connections 2024-09-05T02:00:41.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.181+0000 7f974a751640 1 -- 172.21.15.103:0/1121551275 wait complete. 2024-09-05T02:00:41.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f974a751640 1 Processor -- start 2024-09-05T02:00:41.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f974a751640 1 -- start start 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f974a751640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744100e20 0x7f97441983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f974a751640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102570 0x7f9744198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f974a751640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9744102ef0 0x7f974419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f974a751640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9744110e50 con 0x7f9744102570 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f974a751640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9744110cd0 con 0x7f9744102ef0 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f974a751640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9744110fd0 con 0x7f9744100e20 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f97437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102570 0x7f9744198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f97437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102570 0x7f9744198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55606/0 (socket says 172.21.15.103:55606) 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f97437fe640 1 -- 172.21.15.103:0/4232676070 learned_addr learned my addr 172.21.15.103:0/4232676070 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f9743fff640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744100e20 0x7f97441983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:41.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f9748cee640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9744102ef0 0x7f974419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f97437fe640 1 -- 172.21.15.103:0/4232676070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744100e20 msgr2=0x7f97441983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f97437fe640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744100e20 0x7f97441983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f97437fe640 1 -- 172.21.15.103:0/4232676070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9744102ef0 msgr2=0x7f974419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f97437fe640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9744102ef0 0x7f974419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f97437fe640 1 -- 172.21.15.103:0/4232676070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974419d370 con 0x7f9744102570 2024-09-05T02:00:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f9743fff640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744100e20 0x7f97441983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:00:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.182+0000 7f9748cee640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9744102ef0 0x7f974419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:00:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.183+0000 7f97437fe640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102570 0x7f9744198900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f973800c920 tx=0x7f973800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.183+0000 7f97417fa640 1 -- 172.21.15.103:0/4232676070 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738007c70 con 0x7f9744102570 2024-09-05T02:00:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.183+0000 7f97417fa640 1 -- 172.21.15.103:0/4232676070 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9738007e10 con 0x7f9744102570 2024-09-05T02:00:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.183+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974419d660 con 0x7f9744102570 2024-09-05T02:00:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.183+0000 7f97417fa640 1 -- 172.21.15.103:0/4232676070 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738018820 con 0x7f9744102570 2024-09-05T02:00:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.183+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97441a4f40 con 0x7f9744102570 2024-09-05T02:00:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.184+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9744101200 con 0x7f9744102570 2024-09-05T02:00:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.185+0000 7f97417fa640 1 -- 172.21.15.103:0/4232676070 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f97380189c0 con 0x7f9744102570 2024-09-05T02:00:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.186+0000 7f97417fa640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f972c0a2b80 0x7f972c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.186+0000 7f9743fff640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f972c0a2b80 0x7f972c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.186+0000 7f97417fa640 1 -- 172.21.15.103:0/4232676070 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f97380cf4a0 con 0x7f9744102570 2024-09-05T02:00:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.186+0000 7f9743fff640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f972c0a2b80 0x7f972c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9734009930 tx=0x7f9734008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:41.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.190+0000 7f97417fa640 1 -- 172.21.15.103:0/4232676070 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9738014030 con 0x7f9744102570 2024-09-05T02:00:41.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.365+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f974410a0d0 con 0x7f9744102570 2024-09-05T02:00:41.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.366+0000 7f97417fa640 1 -- 172.21.15.103:0/4232676070 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+601 (secure 0 0 0) 0x7f973809c1c0 con 0x7f9744102570 2024-09-05T02:00:41.367 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2384335573 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:41.367 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2384335573 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:41.367 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2155492495 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:41.367 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/396485172 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:41.367 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1467689006 2024-09-06T01:53:25.001822+0000 2024-09-05T02:00:41.368 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3874104813 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:41.368 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1573405851 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:41.368 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2132077607 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:41.368 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3892241075 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:41.368 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2132077607 2024-09-06T01:53:03.000202+0000 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f972c0a2b80 msgr2=0x7f972c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f972c0a2b80 0x7f972c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9734009930 tx=0x7f9734008040 comp rx=0 tx=0).stop 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102570 msgr2=0x7f9744198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102570 0x7f9744198900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f973800c920 tx=0x7f973800cde0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 shutdown_connections 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9744102ef0 0x7f974419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744102570 0x7f9744198900 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f972c0a2b80 0x7f972c0a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 --2- 172.21.15.103:0/4232676070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744100e20 0x7f97441983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:41.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 >> 172.21.15.103:0/4232676070 conn(0x7f97440fc7d0 msgr2=0x7f974406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:41.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 shutdown_connections 2024-09-05T02:00:41.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:41.368+0000 7f974a751640 1 -- 172.21.15.103:0/4232676070 wait complete. 2024-09-05T02:00:42.001 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi103.front.sepia.ceph.com... 2024-09-05T02:00:42.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:42.002 DEBUG:teuthology.orchestra.run.smithi103:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-05T02:00:42.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi103.stderr:ceph-fuse[72301]: fuse finished with error 0 and tester_r 0 2024-09-05T02:00:42.029 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T02:00:42.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:42 smithi103 ceph-mon[30902]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T02:00:42.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:42 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/4232676070' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:00:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:42 smithi167 ceph-mon[35122]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T02:00:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:42 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/4232676070' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:00:42.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:42 smithi139 ceph-mon[34262]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T02:00:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:42 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/4232676070' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:00:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:44 smithi167 ceph-mon[35122]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T02:00:44.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:44 smithi139 ceph-mon[34262]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T02:00:44.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:44 smithi103 ceph-mon[30902]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T02:00:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:46 smithi167 ceph-mon[35122]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T02:00:46.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:46 smithi139 ceph-mon[34262]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T02:00:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:46 smithi103 ceph-mon[30902]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T02:00:48.132 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-09-05T02:00:48.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:00:48.132 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-05T02:00:48.160 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:00:48.456 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:48.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:48 smithi103 ceph-mon[30902]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T02:00:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:48 smithi167 ceph-mon[35122]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T02:00:48.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:48 smithi139 ceph-mon[34262]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T02:00:48.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 -- 172.21.15.103:0/407925429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14106a90 msgr2=0x7f2d14106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:48.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/407925429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14106a90 0x7f2d14106e70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009cd0 tx=0x7f2d0402a8e0 comp rx=0 tx=0).stop 2024-09-05T02:00:48.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 -- 172.21.15.103:0/407925429 shutdown_connections 2024-09-05T02:00:48.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/407925429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d14101430 0x7f2d1410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/407925429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14100a90 0x7f2d14100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/407925429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14106a90 0x7f2d14106e70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 -- 172.21.15.103:0/407925429 >> 172.21.15.103:0/407925429 conn(0x7f2d140fc7d0 msgr2=0x7f2d140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 -- 172.21.15.103:0/407925429 shutdown_connections 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 -- 172.21.15.103:0/407925429 wait complete. 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 Processor -- start 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.936+0000 7f2d1b8da640 1 -- start start 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d1b8da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d14100a90 0x7f2d14199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d1b8da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14101430 0x7f2d14199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d1b8da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14106a90 0x7f2d141a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d1b8da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14110d40 con 0x7f2d14101430 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d1b8da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14110bc0 con 0x7f2d14100a90 2024-09-05T02:00:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d1b8da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14110ec0 con 0x7f2d14106a90 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d18e75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14101430 0x7f2d14199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d18e75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14101430 0x7f2d14199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55626/0 (socket says 172.21.15.103:55626) 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d18e75640 1 -- 172.21.15.103:0/3125634167 learned_addr learned my addr 172.21.15.103:0/3125634167 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d18e75640 1 -- 172.21.15.103:0/3125634167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14106a90 msgr2=0x7f2d141a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d18e75640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14106a90 0x7f2d141a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d18e75640 1 -- 172.21.15.103:0/3125634167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d14100a90 msgr2=0x7f2d14199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d19676640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d14100a90 0x7f2d14199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d18e75640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d14100a90 0x7f2d14199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d18e75640 1 -- 172.21.15.103:0/3125634167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d141a47f0 con 0x7f2d14101430 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d18e75640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14101430 0x7f2d14199770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800d770 tx=0x7f2d0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.937+0000 7f2d127fc640 1 -- 172.21.15.103:0/3125634167 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08017070 con 0x7f2d14101430 2024-09-05T02:00:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.938+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d141a49c0 con 0x7f2d14101430 2024-09-05T02:00:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.938+0000 7f2d127fc640 1 -- 172.21.15.103:0/3125634167 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d080054d0 con 0x7f2d14101430 2024-09-05T02:00:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.938+0000 7f2d127fc640 1 -- 172.21.15.103:0/3125634167 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08004cf0 con 0x7f2d14101430 2024-09-05T02:00:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.938+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d141a4eb0 con 0x7f2d14101430 2024-09-05T02:00:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.939+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d14100ef0 con 0x7f2d14101430 2024-09-05T02:00:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.940+0000 7f2d127fc640 1 -- 172.21.15.103:0/3125634167 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2d08004e90 con 0x7f2d14101430 2024-09-05T02:00:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.940+0000 7f2d127fc640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2cfc0a2b80 0x7f2cfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.941+0000 7f2d19676640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2cfc0a2b80 0x7f2cfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.941+0000 7f2d127fc640 1 -- 172.21.15.103:0/3125634167 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f2d080ceda0 con 0x7f2d14101430 2024-09-05T02:00:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.941+0000 7f2d19676640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2cfc0a2b80 0x7f2cfc0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006010 tx=0x7f2d04000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:48.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:48.949+0000 7f2d127fc640 1 -- 172.21.15.103:0/3125634167 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d0809bac0 con 0x7f2d14101430 2024-09-05T02:00:49.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.119+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d14109000 con 0x7f2d14101430 2024-09-05T02:00:49.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.122+0000 7f2d127fc640 1 -- 172.21.15.103:0/3125634167 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f2d0809b3c0 con 0x7f2d14101430 2024-09-05T02:00:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:49.125 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:00:24.085875+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:00:49.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2cfc0a2b80 msgr2=0x7f2cfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:49.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2cfc0a2b80 0x7f2cfc0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006010 tx=0x7f2d04000f40 comp rx=0 tx=0).stop 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14101430 msgr2=0x7f2d14199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14101430 0x7f2d14199770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800d770 tx=0x7f2d0800dc30 comp rx=0 tx=0).stop 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 shutdown_connections 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2cfc0a2b80 0x7f2cfc0a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14106a90 0x7f2d141a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d14101430 0x7f2d14199770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 --2- 172.21.15.103:0/3125634167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d14100a90 0x7f2d14199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 >> 172.21.15.103:0/3125634167 conn(0x7f2d140fc7d0 msgr2=0x7f2d140fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 shutdown_connections 2024-09-05T02:00:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:49.125+0000 7f2d1b8da640 1 -- 172.21.15.103:0/3125634167 wait complete. 2024-09-05T02:00:49.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:49 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3125634167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:49.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:49 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3125634167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:49.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:49 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3125634167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:49.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:00:50.178 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:50.454 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:50 smithi103 ceph-mon[30902]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T02:00:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:50 smithi167 ceph-mon[35122]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T02:00:50.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:50 smithi139 ceph-mon[34262]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T02:00:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.694+0000 7f99acdbf640 1 -- 172.21.15.103:0/1840379778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a8106f20 msgr2=0x7f99a810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.694+0000 7f99acdbf640 1 --2- 172.21.15.103:0/1840379778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a8106f20 0x7f99a810eaa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9998009900 tx=0x7f999802a410 comp rx=0 tx=0).stop 2024-09-05T02:00:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.695+0000 7f99acdbf640 1 -- 172.21.15.103:0/1840379778 shutdown_connections 2024-09-05T02:00:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.695+0000 7f99acdbf640 1 --2- 172.21.15.103:0/1840379778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a8106f20 0x7f99a810eaa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:50.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.695+0000 7f99acdbf640 1 --2- 172.21.15.103:0/1840379778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a81027a0 0x7f99a81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:50.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.695+0000 7f99acdbf640 1 --2- 172.21.15.103:0/1840379778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8101df0 0x7f99a81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:50.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.695+0000 7f99acdbf640 1 -- 172.21.15.103:0/1840379778 >> 172.21.15.103:0/1840379778 conn(0x7f99a80fd8f0 msgr2=0x7f99a80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:50.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.695+0000 7f99acdbf640 1 -- 172.21.15.103:0/1840379778 shutdown_connections 2024-09-05T02:00:50.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.695+0000 7f99acdbf640 1 -- 172.21.15.103:0/1840379778 wait complete. 2024-09-05T02:00:50.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.695+0000 7f99acdbf640 1 Processor -- start 2024-09-05T02:00:50.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.695+0000 7f99acdbf640 1 -- start start 2024-09-05T02:00:50.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99acdbf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8101df0 0x7f99a819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:50.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99acdbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a81027a0 0x7f99a819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:50.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99acdbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a8106f20 0x7f99a819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:50.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99acdbf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8111ea0 con 0x7f99a81027a0 2024-09-05T02:00:50.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99acdbf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8111d20 con 0x7f99a8106f20 2024-09-05T02:00:50.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99acdbf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112020 con 0x7f99a8101df0 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a81027a0 0x7f99a819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a81027a0 0x7f99a819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35606/0 (socket says 172.21.15.103:35606) 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a5d9b640 1 -- 172.21.15.103:0/3400071757 learned_addr learned my addr 172.21.15.103:0/3400071757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a659c640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8101df0 0x7f99a819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a5d9b640 1 -- 172.21.15.103:0/3400071757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8101df0 msgr2=0x7f99a819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a5d9b640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8101df0 0x7f99a819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a5d9b640 1 -- 172.21.15.103:0/3400071757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a8106f20 msgr2=0x7f99a819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a6d9d640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a8106f20 0x7f99a819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a5d9b640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a8106f20 0x7f99a819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a5d9b640 1 -- 172.21.15.103:0/3400071757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a819f530 con 0x7f99a81027a0 2024-09-05T02:00:50.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a659c640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8101df0 0x7f99a819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:00:50.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a6d9d640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a8106f20 0x7f99a819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:00:50.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99a5d9b640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a81027a0 0x7f99a819aac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f999c00ba50 tx=0x7f999c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:50.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99977fe640 1 -- 172.21.15.103:0/3400071757 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c019070 con 0x7f99a81027a0 2024-09-05T02:00:50.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99977fe640 1 -- 172.21.15.103:0/3400071757 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f999c00cbe0 con 0x7f99a81027a0 2024-09-05T02:00:50.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99977fe640 1 -- 172.21.15.103:0/3400071757 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c0075d0 con 0x7f99a81027a0 2024-09-05T02:00:50.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.696+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a819f820 con 0x7f99a81027a0 2024-09-05T02:00:50.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.697+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a7100 con 0x7f99a81027a0 2024-09-05T02:00:50.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.698+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a81021d0 con 0x7f99a81027a0 2024-09-05T02:00:50.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.699+0000 7f99977fe640 1 -- 172.21.15.103:0/3400071757 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f999c01c070 con 0x7f99a81027a0 2024-09-05T02:00:50.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.699+0000 7f99977fe640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c00 0x7f99900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:50.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.699+0000 7f99a659c640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c00 0x7f99900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:50.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.699+0000 7f99977fe640 1 -- 172.21.15.103:0/3400071757 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f999c0ce8e0 con 0x7f99a81027a0 2024-09-05T02:00:50.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.700+0000 7f99a659c640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c00 0x7f99900a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f99a00058c0 tx=0x7f99a0005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:50.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.704+0000 7f99977fe640 1 -- 172.21.15.103:0/3400071757 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f999c09b600 con 0x7f99a81027a0 2024-09-05T02:00:50.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.871+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99a81a7390 con 0x7f99a81027a0 2024-09-05T02:00:50.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.872+0000 7f99977fe640 1 -- 172.21.15.103:0/3400071757 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f999c09af00 con 0x7f99a81027a0 2024-09-05T02:00:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:00:50.874 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:00:24.085875+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:00:50.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c00 msgr2=0x7f99900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:50.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c00 0x7f99900a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f99a00058c0 tx=0x7f99a0005d30 comp rx=0 tx=0).stop 2024-09-05T02:00:50.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a81027a0 msgr2=0x7f99a819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:50.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a81027a0 0x7f99a819aac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f999c00ba50 tx=0x7f999c00bf10 comp rx=0 tx=0).stop 2024-09-05T02:00:50.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 shutdown_connections 2024-09-05T02:00:50.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a8106f20 0x7f99a819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:50.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a81027a0 0x7f99a819aac0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:50.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c00 0x7f99900a50c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:50.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 --2- 172.21.15.103:0/3400071757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8101df0 0x7f99a819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:50.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 >> 172.21.15.103:0/3400071757 conn(0x7f99a80fd8f0 msgr2=0x7f99a80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:50.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 shutdown_connections 2024-09-05T02:00:50.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:50.875+0000 7f99acdbf640 1 -- 172.21.15.103:0/3400071757 wait complete. 2024-09-05T02:00:51.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:51 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3400071757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:51.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:51 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3400071757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:51 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3400071757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:00:51.693 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-05T02:00:51.947 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:52.344 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:52 smithi103 ceph-mon[30902]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.425+0000 7faae8593640 1 -- 172.21.15.103:0/4192087083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0100e20 msgr2=0x7faae0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.425+0000 7faae8593640 1 --2- 172.21.15.103:0/4192087083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0100e20 0x7faae0101200 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faadc009900 tx=0x7faadc02a480 comp rx=0 tx=0).stop 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.425+0000 7faae8593640 1 -- 172.21.15.103:0/4192087083 shutdown_connections 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.425+0000 7faae8593640 1 --2- 172.21.15.103:0/4192087083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae0102ef0 0x7faae010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.425+0000 7faae8593640 1 --2- 172.21.15.103:0/4192087083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faae0102570 0x7faae01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.425+0000 7faae8593640 1 --2- 172.21.15.103:0/4192087083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0100e20 0x7faae0101200 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.425+0000 7faae8593640 1 -- 172.21.15.103:0/4192087083 >> 172.21.15.103:0/4192087083 conn(0x7faae00fc7d0 msgr2=0x7faae00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.425+0000 7faae8593640 1 -- 172.21.15.103:0/4192087083 shutdown_connections 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.425+0000 7faae8593640 1 -- 172.21.15.103:0/4192087083 wait complete. 2024-09-05T02:00:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae8593640 1 Processor -- start 2024-09-05T02:00:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae8593640 1 -- start start 2024-09-05T02:00:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae8593640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae0100e20 0x7faae0197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae8593640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faae0102570 0x7faae0197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae8593640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0102ef0 0x7faae019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae8593640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faae0110e30 con 0x7faae0102ef0 2024-09-05T02:00:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae8593640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faae0110cb0 con 0x7faae0102570 2024-09-05T02:00:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae8593640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faae0110fb0 con 0x7faae0100e20 2024-09-05T02:00:52.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae6b30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0102ef0 0x7faae019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:52.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae6b30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0102ef0 0x7faae019bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35614/0 (socket says 172.21.15.103:35614) 2024-09-05T02:00:52.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae6b30640 1 -- 172.21.15.103:0/3668317525 learned_addr learned my addr 172.21.15.103:0/3668317525 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:52.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.426+0000 7faae5b2e640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faae0102570 0x7faae0197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:52.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae632f640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae0100e20 0x7faae0197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:52.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae6b30640 1 -- 172.21.15.103:0/3668317525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae0100e20 msgr2=0x7faae0197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:52.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae6b30640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae0100e20 0x7faae0197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:52.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae6b30640 1 -- 172.21.15.103:0/3668317525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faae0102570 msgr2=0x7faae0197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:52.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae6b30640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faae0102570 0x7faae0197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:52.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae6b30640 1 -- 172.21.15.103:0/3668317525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae019c2e0 con 0x7faae0102ef0 2024-09-05T02:00:52.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae632f640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae0100e20 0x7faae0197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:00:52.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae6b30640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0102ef0 0x7faae019bc00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faad800eaa0 tx=0x7faad800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:52.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faad37fe640 1 -- 172.21.15.103:0/3668317525 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad800cdb0 con 0x7faae0102ef0 2024-09-05T02:00:52.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faad37fe640 1 -- 172.21.15.103:0/3668317525 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faad8005020 con 0x7faae0102ef0 2024-09-05T02:00:52.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae019c5d0 con 0x7faae0102ef0 2024-09-05T02:00:52.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faad37fe640 1 -- 172.21.15.103:0/3668317525 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad80116d0 con 0x7faae0102ef0 2024-09-05T02:00:52.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.427+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae01a3eb0 con 0x7faae0102ef0 2024-09-05T02:00:52.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.428+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae0101200 con 0x7faae0102ef0 2024-09-05T02:00:52.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.429+0000 7faad37fe640 1 -- 172.21.15.103:0/3668317525 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7faad80051d0 con 0x7faae0102ef0 2024-09-05T02:00:52.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.430+0000 7faad37fe640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faacc0a2b80 0x7faacc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:52.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.430+0000 7faae632f640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faacc0a2b80 0x7faacc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:52.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.430+0000 7faad37fe640 1 -- 172.21.15.103:0/3668317525 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7faad80cf780 con 0x7faae0102ef0 2024-09-05T02:00:52.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.430+0000 7faae632f640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faacc0a2b80 0x7faacc0a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faadc005ab0 tx=0x7faadc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:52.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.433+0000 7faad37fe640 1 -- 172.21.15.103:0/3668317525 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faad8016030 con 0x7faae0102ef0 2024-09-05T02:00:52.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:52.633+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faae0109090 con 0x7faae0102ef0 2024-09-05T02:00:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:52 smithi167 ceph-mon[35122]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:52.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:52 smithi139 ceph-mon[34262]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.178+0000 7faad37fe640 1 -- 172.21.15.103:0/3668317525 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7faad809c4a0 con 0x7faae0102ef0 2024-09-05T02:00:53.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faacc0a2b80 msgr2=0x7faacc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:53.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faacc0a2b80 0x7faacc0a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faadc005ab0 tx=0x7faadc005a20 comp rx=0 tx=0).stop 2024-09-05T02:00:53.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0102ef0 msgr2=0x7faae019bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:53.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0102ef0 0x7faae019bc00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faad800eaa0 tx=0x7faad800ef60 comp rx=0 tx=0).stop 2024-09-05T02:00:53.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 shutdown_connections 2024-09-05T02:00:53.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0102ef0 0x7faae019bc00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:53.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faae0102570 0x7faae0197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:53.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faacc0a2b80 0x7faacc0a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:53.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 --2- 172.21.15.103:0/3668317525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae0100e20 0x7faae0197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:53.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.180+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 >> 172.21.15.103:0/3668317525 conn(0x7faae00fc7d0 msgr2=0x7faae00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:53.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.181+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 shutdown_connections 2024-09-05T02:00:53.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:53.181+0000 7faae8593640 1 -- 172.21.15.103:0/3668317525 wait complete. 2024-09-05T02:00:53.566 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:53 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3668317525' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T02:00:53.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:53 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3668317525' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T02:00:53.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:53 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3668317525' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T02:00:53.982 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-05T02:00:53.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-05T02:00:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:54.350 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:54 smithi103 ceph-mon[30902]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:54.350 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:54 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3668317525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T02:00:54.350 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:54 smithi103 ceph-mon[30902]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:54 smithi167 ceph-mon[35122]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:54 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3668317525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T02:00:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:54 smithi167 ceph-mon[35122]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:54 smithi139 ceph-mon[34262]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:54.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:54 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3668317525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T02:00:54.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:54 smithi139 ceph-mon[34262]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:54.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.720+0000 7f37933da640 1 -- 172.21.15.103:0/3331423000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 msgr2=0x7f378c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:54.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.720+0000 7f37933da640 1 --2- 172.21.15.103:0/3331423000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 0x7f378c1067e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f377c009900 tx=0x7f377c02a410 comp rx=0 tx=0).stop 2024-09-05T02:00:54.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.720+0000 7f37933da640 1 -- 172.21.15.103:0/3331423000 shutdown_connections 2024-09-05T02:00:54.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.720+0000 7f37933da640 1 --2- 172.21.15.103:0/3331423000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f378c106f10 0x7f378c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:54.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.720+0000 7f37933da640 1 --2- 172.21.15.103:0/3331423000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 0x7f378c1067e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:54.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.720+0000 7f37933da640 1 --2- 172.21.15.103:0/3331423000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378c101de0 0x7f378c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:54.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.720+0000 7f37933da640 1 -- 172.21.15.103:0/3331423000 >> 172.21.15.103:0/3331423000 conn(0x7f378c0fd8c0 msgr2=0x7f378c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:54.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.720+0000 7f37933da640 1 -- 172.21.15.103:0/3331423000 shutdown_connections 2024-09-05T02:00:54.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.720+0000 7f37933da640 1 -- 172.21.15.103:0/3331423000 wait complete. 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f37933da640 1 Processor -- start 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f37933da640 1 -- start start 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f37933da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f378c101de0 0x7f378c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f37933da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 0x7f378c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f37933da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378c106f10 0x7f378c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f37933da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f378c111e40 con 0x7f378c102790 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f37933da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f378c111cc0 con 0x7f378c101de0 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f37933da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f378c111fc0 con 0x7f378c106f10 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f3790975640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 0x7f378c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f3790975640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 0x7f378c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35638/0 (socket says 172.21.15.103:35638) 2024-09-05T02:00:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f3790975640 1 -- 172.21.15.103:0/2562504247 learned_addr learned my addr 172.21.15.103:0/2562504247 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.721+0000 7f3791176640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f378c101de0 0x7f378c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f3791977640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378c106f10 0x7f378c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f3790975640 1 -- 172.21.15.103:0/2562504247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378c106f10 msgr2=0x7f378c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f3790975640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378c106f10 0x7f378c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f3790975640 1 -- 172.21.15.103:0/2562504247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f378c101de0 msgr2=0x7f378c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f3790975640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f378c101de0 0x7f378c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f3790975640 1 -- 172.21.15.103:0/2562504247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c1a6900 con 0x7f378c102790 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f3790975640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 0x7f378c074a30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f377c006010 tx=0x7f377c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f37827fc640 1 -- 172.21.15.103:0/2562504247 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c03f030 con 0x7f378c102790 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f37827fc640 1 -- 172.21.15.103:0/2562504247 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f377c0035f0 con 0x7f378c102790 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378c1a6b90 con 0x7f378c102790 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f37827fc640 1 -- 172.21.15.103:0/2562504247 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c035070 con 0x7f378c102790 2024-09-05T02:00:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.722+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378c1a7070 con 0x7f378c102790 2024-09-05T02:00:54.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.723+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378c1021c0 con 0x7f378c102790 2024-09-05T02:00:54.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.724+0000 7f37827fc640 1 -- 172.21.15.103:0/2562504247 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f377c003b10 con 0x7f378c102790 2024-09-05T02:00:54.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.725+0000 7f37827fc640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37780a2b80 0x7f37780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:54.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.725+0000 7f3791176640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37780a2b80 0x7f37780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:54.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.725+0000 7f37827fc640 1 -- 172.21.15.103:0/2562504247 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f377c0efd20 con 0x7f378c102790 2024-09-05T02:00:54.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.725+0000 7f3791176640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37780a2b80 0x7f37780a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3788009990 tx=0x7f3788009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:54.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.729+0000 7f37827fc640 1 -- 172.21.15.103:0/2562504247 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f377c0bca40 con 0x7f378c102790 2024-09-05T02:00:54.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:54.931+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f378c10a150 con 0x7f378c102790 2024-09-05T02:00:55.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.185+0000 7f37827fc640 1 -- 172.21.15.103:0/2562504247 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f377c0bc340 con 0x7f378c102790 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37780a2b80 msgr2=0x7f37780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37780a2b80 0x7f37780a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3788009990 tx=0x7f3788009110 comp rx=0 tx=0).stop 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 msgr2=0x7f378c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 0x7f378c074a30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f377c006010 tx=0x7f377c004270 comp rx=0 tx=0).stop 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 shutdown_connections 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37780a2b80 0x7f37780a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378c106f10 0x7f378c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f378c102790 0x7f378c074a30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 --2- 172.21.15.103:0/2562504247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f378c101de0 0x7f378c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 >> 172.21.15.103:0/2562504247 conn(0x7f378c0fd8c0 msgr2=0x7f378c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 shutdown_connections 2024-09-05T02:00:55.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:55.187+0000 7f37933da640 1 -- 172.21.15.103:0/2562504247 wait complete. 2024-09-05T02:00:55.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:55 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2562504247' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-05T02:00:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:55 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2562504247' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-05T02:00:55.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:55 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2562504247' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-05T02:00:55.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-05T02:00:56.194 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:56.439 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:56 smithi103 ceph-mon[30902]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:56.439 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:56 smithi103 ceph-mon[30902]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T02:00:56.440 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:56 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2562504247' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-05T02:00:56.440 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:56 smithi103 ceph-mon[30902]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T02:00:56.440 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:56 smithi103 ceph-mon[30902]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:56.440 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:56 smithi103 ceph-mon[30902]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-05T02:00:56.440 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:56 smithi103 ceph-mon[30902]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-05T02:00:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:56 smithi167 ceph-mon[35122]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:56 smithi167 ceph-mon[35122]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T02:00:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:56 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2562504247' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-05T02:00:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:56 smithi167 ceph-mon[35122]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T02:00:56.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:56 smithi167 ceph-mon[35122]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:56.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:56 smithi167 ceph-mon[35122]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-05T02:00:56.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:56 smithi167 ceph-mon[35122]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-05T02:00:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:56 smithi139 ceph-mon[34262]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:56.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:56 smithi139 ceph-mon[34262]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T02:00:56.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:56 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2562504247' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-05T02:00:56.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:56 smithi139 ceph-mon[34262]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T02:00:56.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:56 smithi139 ceph-mon[34262]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T02:00:56.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:56 smithi139 ceph-mon[34262]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-05T02:00:56.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:56 smithi139 ceph-mon[34262]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-05T02:00:56.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.647+0000 7ff916e33640 1 -- 172.21.15.103:0/2573914591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910101420 msgr2=0x7ff91010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:56.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.647+0000 7ff916e33640 1 --2- 172.21.15.103:0/2573914591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910101420 0x7ff91010d950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff908009cd0 tx=0x7ff90802a8e0 comp rx=0 tx=0).stop 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.647+0000 7ff916e33640 1 -- 172.21.15.103:0/2573914591 shutdown_connections 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.647+0000 7ff916e33640 1 --2- 172.21.15.103:0/2573914591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910101420 0x7ff91010d950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.647+0000 7ff916e33640 1 --2- 172.21.15.103:0/2573914591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff910100a80 0x7ff910100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.647+0000 7ff916e33640 1 --2- 172.21.15.103:0/2573914591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff910106a80 0x7ff910106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.647+0000 7ff916e33640 1 -- 172.21.15.103:0/2573914591 >> 172.21.15.103:0/2573914591 conn(0x7ff9100fc7a0 msgr2=0x7ff9100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.647+0000 7ff916e33640 1 -- 172.21.15.103:0/2573914591 shutdown_connections 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.647+0000 7ff916e33640 1 -- 172.21.15.103:0/2573914591 wait complete. 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff916e33640 1 Processor -- start 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff916e33640 1 -- start start 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff916e33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910100a80 0x7ff9101982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff916e33640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff910101420 0x7ff910198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff916e33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff910106a80 0x7ff91019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff916e33640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff910111e50 con 0x7ff910100a80 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff916e33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff910111cd0 con 0x7ff910106a80 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff916e33640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff910111fd0 con 0x7ff910101420 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff914bcf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910100a80 0x7ff9101982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff914bcf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910100a80 0x7ff9101982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35666/0 (socket says 172.21.15.103:35666) 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff914bcf640 1 -- 172.21.15.103:0/2794778690 learned_addr learned my addr 172.21.15.103:0/2794778690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff90ffff640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff910101420 0x7ff910198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff9153d0640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff910106a80 0x7ff91019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff914bcf640 1 -- 172.21.15.103:0/2794778690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff910101420 msgr2=0x7ff910198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff914bcf640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff910101420 0x7ff910198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff914bcf640 1 -- 172.21.15.103:0/2794778690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff910106a80 msgr2=0x7ff91019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:56.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff914bcf640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff910106a80 0x7ff91019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff914bcf640 1 -- 172.21.15.103:0/2794778690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91019d2a0 con 0x7ff910100a80 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.648+0000 7ff90ffff640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff910101420 0x7ff910198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.649+0000 7ff914bcf640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910100a80 0x7ff9101982f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff90400d770 tx=0x7ff90400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.649+0000 7ff90dffb640 1 -- 172.21.15.103:0/2794778690 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904017070 con 0x7ff910100a80 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.649+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91019d590 con 0x7ff910100a80 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.649+0000 7ff90dffb640 1 -- 172.21.15.103:0/2794778690 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9040054d0 con 0x7ff910100a80 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.649+0000 7ff90dffb640 1 -- 172.21.15.103:0/2794778690 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904004cf0 con 0x7ff910100a80 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.649+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101a4e70 con 0x7ff910100a80 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.650+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff910100ee0 con 0x7ff910100a80 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.651+0000 7ff90dffb640 1 -- 172.21.15.103:0/2794778690 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff904004e90 con 0x7ff910100a80 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.651+0000 7ff90dffb640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9000a2b80 0x7ff9000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.652+0000 7ff90ffff640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9000a2b80 0x7ff9000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.652+0000 7ff90dffb640 1 -- 172.21.15.103:0/2794778690 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7ff9040ced00 con 0x7ff910100a80 2024-09-05T02:00:56.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.652+0000 7ff90ffff640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9000a2b80 0x7ff9000a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff9100fedc0 tx=0x7ff8fc00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:56.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.656+0000 7ff90dffb640 1 -- 172.21.15.103:0/2794778690 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff90409ba20 con 0x7ff910100a80 2024-09-05T02:00:56.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:56.857+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff910108ff0 con 0x7ff910100a80 2024-09-05T02:00:57.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.199+0000 7ff90dffb640 1 -- 172.21.15.103:0/2794778690 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7ff90409b320 con 0x7ff910100a80 2024-09-05T02:00:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9000a2b80 msgr2=0x7ff9000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9000a2b80 0x7ff9000a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff9100fedc0 tx=0x7ff8fc00a5c0 comp rx=0 tx=0).stop 2024-09-05T02:00:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910100a80 msgr2=0x7ff9101982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910100a80 0x7ff9101982f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff90400d770 tx=0x7ff90400dc30 comp rx=0 tx=0).stop 2024-09-05T02:00:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 shutdown_connections 2024-09-05T02:00:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff910106a80 0x7ff91019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9000a2b80 0x7ff9000a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff910101420 0x7ff910198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 --2- 172.21.15.103:0/2794778690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff910100a80 0x7ff9101982f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:57.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 >> 172.21.15.103:0/2794778690 conn(0x7ff9100fc7a0 msgr2=0x7ff9100fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:57.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 shutdown_connections 2024-09-05T02:00:57.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:57.201+0000 7ff916e33640 1 -- 172.21.15.103:0/2794778690 wait complete. 2024-09-05T02:00:57.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:57 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] up:active 2024-09-05T02:00:57.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:57 smithi103 ceph-mon[30902]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T02:00:57.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:57 smithi103 ceph-mon[30902]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-05T02:00:57.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:57 smithi103 ceph-mon[30902]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-05T02:00:57.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:57 smithi103 ceph-mon[30902]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-05T02:00:57.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:57 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2794778690' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T02:00:57.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:57 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] up:active 2024-09-05T02:00:57.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:57 smithi167 ceph-mon[35122]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T02:00:57.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:57 smithi167 ceph-mon[35122]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-05T02:00:57.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:57 smithi167 ceph-mon[35122]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-05T02:00:57.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:57 smithi167 ceph-mon[35122]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-05T02:00:57.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:57 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2794778690' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T02:00:57.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:57 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] up:active 2024-09-05T02:00:57.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:57 smithi139 ceph-mon[34262]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T02:00:57.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:57 smithi139 ceph-mon[34262]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-05T02:00:57.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:57 smithi139 ceph-mon[34262]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-05T02:00:57.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:57 smithi139 ceph-mon[34262]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-05T02:00:57.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:57 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2794778690' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T02:00:57.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-05T02:00:58.205 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:00:58.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:58 smithi103 ceph-mon[30902]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:58.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:58 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2794778690' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T02:00:58.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:58 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] up:active 2024-09-05T02:00:58.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:58 smithi103 ceph-mon[30902]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-05T02:00:58.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:58 smithi103 ceph-mon[30902]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-05T02:00:58.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:58 smithi103 ceph-mon[30902]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-05T02:00:58.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:58 smithi103 ceph-mon[30902]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-05T02:00:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:58 smithi167 ceph-mon[35122]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:58 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2794778690' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T02:00:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:58 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] up:active 2024-09-05T02:00:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:58 smithi167 ceph-mon[35122]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-05T02:00:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:58 smithi167 ceph-mon[35122]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-05T02:00:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:58 smithi167 ceph-mon[35122]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-05T02:00:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:58 smithi167 ceph-mon[35122]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-05T02:00:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:58 smithi139 ceph-mon[34262]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T02:00:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:58 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2794778690' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T02:00:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:58 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] up:active 2024-09-05T02:00:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:58 smithi139 ceph-mon[34262]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-05T02:00:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:58 smithi139 ceph-mon[34262]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-05T02:00:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:58 smithi139 ceph-mon[34262]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-05T02:00:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:58 smithi139 ceph-mon[34262]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-05T02:00:58.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.723+0000 7fe9952d5640 1 -- 172.21.15.103:0/2036059092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe99010c180 msgr2=0x7fe99010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:58.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.723+0000 7fe9952d5640 1 --2- 172.21.15.103:0/2036059092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe99010c180 0x7fe99010e610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe988009ca0 tx=0x7fe98802a950 comp rx=0 tx=0).stop 2024-09-05T02:00:58.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.723+0000 7fe9952d5640 1 -- 172.21.15.103:0/2036059092 shutdown_connections 2024-09-05T02:00:58.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.723+0000 7fe9952d5640 1 --2- 172.21.15.103:0/2036059092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe99010c180 0x7fe99010e610 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:58.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.723+0000 7fe9952d5640 1 --2- 172.21.15.103:0/2036059092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe990103700 0x7fe99010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.723+0000 7fe9952d5640 1 --2- 172.21.15.103:0/2036059092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe990102de0 0x7fe9901031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.723+0000 7fe9952d5640 1 -- 172.21.15.103:0/2036059092 >> 172.21.15.103:0/2036059092 conn(0x7fe9900fc7a0 msgr2=0x7fe9900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.723+0000 7fe9952d5640 1 -- 172.21.15.103:0/2036059092 shutdown_connections 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.723+0000 7fe9952d5640 1 -- 172.21.15.103:0/2036059092 wait complete. 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe9952d5640 1 Processor -- start 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe9952d5640 1 -- start start 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe9952d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe990102de0 0x7fe9901982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe9952d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe990103700 0x7fe9901987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe9952d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe99010c180 0x7fe99019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe9952d5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe990111de0 con 0x7fe990102de0 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe9952d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe990111c60 con 0x7fe990103700 2024-09-05T02:00:58.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe9952d5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe990111f60 con 0x7fe99010c180 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe98effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe990102de0 0x7fe9901982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe98effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe990102de0 0x7fe9901982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35690/0 (socket says 172.21.15.103:35690) 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe98effd640 1 -- 172.21.15.103:0/1842250120 learned_addr learned my addr 172.21.15.103:0/1842250120 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe98f7fe640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe99010c180 0x7fe99019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.724+0000 7fe98e7fc640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe990103700 0x7fe9901987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe98effd640 1 -- 172.21.15.103:0/1842250120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe99010c180 msgr2=0x7fe99019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe98effd640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe99010c180 0x7fe99019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe98effd640 1 -- 172.21.15.103:0/1842250120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe990103700 msgr2=0x7fe9901987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe98effd640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe990103700 0x7fe9901987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe98effd640 1 -- 172.21.15.103:0/1842250120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99019d250 con 0x7fe990102de0 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe98effd640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe990102de0 0x7fe9901982a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c920 tx=0x7fe98000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe977fff640 1 -- 172.21.15.103:0/1842250120 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980007c70 con 0x7fe990102de0 2024-09-05T02:00:58.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe977fff640 1 -- 172.21.15.103:0/1842250120 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe980007e10 con 0x7fe990102de0 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99019d540 con 0x7fe990102de0 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe977fff640 1 -- 172.21.15.103:0/1842250120 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980018820 con 0x7fe990102de0 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.725+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9901a4e20 con 0x7fe990102de0 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.726+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9901031c0 con 0x7fe990102de0 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.727+0000 7fe977fff640 1 -- 172.21.15.103:0/1842250120 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe9800189c0 con 0x7fe990102de0 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.728+0000 7fe977fff640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9780a2c00 0x7fe9780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.728+0000 7fe98e7fc640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9780a2c00 0x7fe9780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.728+0000 7fe977fff640 1 -- 172.21.15.103:0/1842250120 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fe9800cf350 con 0x7fe990102de0 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.728+0000 7fe98e7fc640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9780a2c00 0x7fe9780a50c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe984005d90 tx=0x7fe984005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:00:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.732+0000 7fe977fff640 1 -- 172.21.15.103:0/1842250120 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe980014030 con 0x7fe990102de0 2024-09-05T02:00:58.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:58.934+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fe990199600 con 0x7fe990102de0 2024-09-05T02:00:59.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.228+0000 7fe977fff640 1 -- 172.21.15.103:0/1842250120 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fe98009c070 con 0x7fe990102de0 2024-09-05T02:00:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9780a2c00 msgr2=0x7fe9780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9780a2c00 0x7fe9780a50c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe984005d90 tx=0x7fe984005d20 comp rx=0 tx=0).stop 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe990102de0 msgr2=0x7fe9901982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe990102de0 0x7fe9901982a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c920 tx=0x7fe98000cde0 comp rx=0 tx=0).stop 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 shutdown_connections 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9780a2c00 0x7fe9780a50c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe99010c180 0x7fe99019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe990103700 0x7fe9901987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 --2- 172.21.15.103:0/1842250120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe990102de0 0x7fe9901982a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 >> 172.21.15.103:0/1842250120 conn(0x7fe9900fc7a0 msgr2=0x7fe9900fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 shutdown_connections 2024-09-05T02:00:59.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:00:59.230+0000 7fe9952d5640 1 -- 172.21.15.103:0/1842250120 wait complete. 2024-09-05T02:00:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] up:active 2024-09-05T02:00:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-mon[34262]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-05T02:00:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-mon[34262]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T02:00:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-mon[34262]: Cluster is now healthy 2024-09-05T02:00:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-mon[34262]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-05T02:00:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-mon[34262]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-05T02:00:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-mon[34262]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-05T02:00:59.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1842250120' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T02:00:59.397 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b[65040]: 2024-09-05T02:00:59.097+0000 7f1baa8ff640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-05T02:00:59.397 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:00:59 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b[65040]: 2024-09-05T02:00:59.097+0000 7f1baa8ff640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-05T02:00:59.590 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:59 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] up:active 2024-09-05T02:00:59.591 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:59 smithi103 ceph-mon[30902]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-05T02:00:59.591 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:59 smithi103 ceph-mon[30902]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T02:00:59.591 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:59 smithi103 ceph-mon[30902]: Cluster is now healthy 2024-09-05T02:00:59.591 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:59 smithi103 ceph-mon[30902]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-05T02:00:59.591 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:59 smithi103 ceph-mon[30902]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-05T02:00:59.591 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:59 smithi103 ceph-mon[30902]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-05T02:00:59.591 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:00:59 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1842250120' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T02:00:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:59 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] up:active 2024-09-05T02:00:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:59 smithi167 ceph-mon[35122]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-05T02:00:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:59 smithi167 ceph-mon[35122]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T02:00:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:59 smithi167 ceph-mon[35122]: Cluster is now healthy 2024-09-05T02:00:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:59 smithi167 ceph-mon[35122]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-05T02:00:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:59 smithi167 ceph-mon[35122]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-05T02:00:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:59 smithi167 ceph-mon[35122]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-05T02:00:59.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:00:59 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1842250120' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T02:00:59.937 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-05T02:00:59.937 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-05T02:01:00.186 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:00.457 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-mon[30902]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-05T02:01:00.457 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1842250120' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T02:01:00.457 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] up:active 2024-09-05T02:01:00.457 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-mon[30902]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-05T02:01:00.457 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:01:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:00 smithi167 ceph-mon[35122]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-05T02:01:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:00 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1842250120' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T02:01:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:00 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] up:active 2024-09-05T02:01:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:00 smithi167 ceph-mon[35122]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-05T02:01:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:00 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:01:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:00 smithi139 ceph-mon[34262]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-05T02:01:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:00 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1842250120' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T02:01:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:00 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] up:active 2024-09-05T02:01:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:00 smithi139 ceph-mon[34262]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-05T02:01:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:00 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:01:00.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 -- 172.21.15.103:0/4109439065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 msgr2=0x7f1cb0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:00.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 --2- 172.21.15.103:0/4109439065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 0x7f1cb0101200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009900 tx=0x7f1ca002a480 comp rx=0 tx=0).stop 2024-09-05T02:01:00.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 -- 172.21.15.103:0/4109439065 shutdown_connections 2024-09-05T02:01:00.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 --2- 172.21.15.103:0/4109439065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb0102ef0 0x7f1cb010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 --2- 172.21.15.103:0/4109439065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cb0102570 0x7f1cb01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 --2- 172.21.15.103:0/4109439065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 0x7f1cb0101200 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 -- 172.21.15.103:0/4109439065 >> 172.21.15.103:0/4109439065 conn(0x7f1cb00fc7d0 msgr2=0x7f1cb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:00.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 -- 172.21.15.103:0/4109439065 shutdown_connections 2024-09-05T02:01:00.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 -- 172.21.15.103:0/4109439065 wait complete. 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.695+0000 7f1cb5644640 1 Processor -- start 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1cb5644640 1 -- start start 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1cb5644640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 0x7f1cb010feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1cb5644640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cb0102570 0x7f1cb01103f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1cb5644640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb0102ef0 0x7f1cb0110930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1cb5644640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0111ff0 con 0x7f1cb0100e20 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1cb5644640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb006ae60 con 0x7f1cb0102570 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1cb5644640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0112170 con 0x7f1cb0102ef0 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 0x7f1cb010feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1caeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 0x7f1cb010feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54848/0 (socket says 172.21.15.103:54848) 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1caeffd640 1 -- 172.21.15.103:0/1038018705 learned_addr learned my addr 172.21.15.103:0/1038018705 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:00.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1cae7fc640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cb0102570 0x7f1cb01103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1caeffd640 1 -- 172.21.15.103:0/1038018705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb0102ef0 msgr2=0x7f1cb0110930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1caeffd640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb0102ef0 0x7f1cb0110930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1caeffd640 1 -- 172.21.15.103:0/1038018705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cb0102570 msgr2=0x7f1cb01103f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1caeffd640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cb0102570 0x7f1cb01103f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1caeffd640 1 -- 172.21.15.103:0/1038018705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb01a6930 con 0x7f1cb0100e20 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.696+0000 7f1caeffd640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 0x7f1cb010feb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0006010 tx=0x7f1ca00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.697+0000 7f1c97fff640 1 -- 172.21.15.103:0/1038018705 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca003f070 con 0x7f1cb0100e20 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.697+0000 7f1c97fff640 1 -- 172.21.15.103:0/1038018705 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ca0003aa0 con 0x7f1cb0100e20 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.697+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb01a6bc0 con 0x7f1cb0100e20 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.697+0000 7f1c97fff640 1 -- 172.21.15.103:0/1038018705 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0035030 con 0x7f1cb0100e20 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.697+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb01a70a0 con 0x7f1cb0100e20 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.698+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb0101200 con 0x7f1cb0100e20 2024-09-05T02:01:00.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.699+0000 7f1c97fff640 1 -- 172.21.15.103:0/1038018705 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1ca0042070 con 0x7f1cb0100e20 2024-09-05T02:01:00.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.700+0000 7f1c97fff640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1c980a2b80 0x7f1c980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:00.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.700+0000 7f1cae7fc640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1c980a2b80 0x7f1c980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:00.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.700+0000 7f1c97fff640 1 -- 172.21.15.103:0/1038018705 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f1ca00efed0 con 0x7f1cb0100e20 2024-09-05T02:01:00.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.700+0000 7f1cae7fc640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1c980a2b80 0x7f1c980a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4005e10 tx=0x7f1ca4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:00.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.703+0000 7f1c97fff640 1 -- 172.21.15.103:0/1038018705 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ca00bcbf0 con 0x7f1cb0100e20 2024-09-05T02:01:00.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.894+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1cb0109100 con 0x7f1c980a2b80 2024-09-05T02:01:00.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.922+0000 7f1c97fff640 1 -- 172.21.15.103:0/1038018705 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1ca400eac0 con 0x7f1c980a2b80 2024-09-05T02:01:00.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1c980a2b80 msgr2=0x7f1c980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:00.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1c980a2b80 0x7f1c980a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4005e10 tx=0x7f1ca4005da0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 msgr2=0x7f1cb010feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:00.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 0x7f1cb010feb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0006010 tx=0x7f1ca00042f0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 shutdown_connections 2024-09-05T02:01:00.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1c980a2b80 0x7f1c980a5040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb0102ef0 0x7f1cb0110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1cb0102570 0x7f1cb01103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 --2- 172.21.15.103:0/1038018705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cb0100e20 0x7f1cb010feb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:00.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 >> 172.21.15.103:0/1038018705 conn(0x7f1cb00fc7d0 msgr2=0x7f1cb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:00.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 shutdown_connections 2024-09-05T02:01:00.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:00.924+0000 7f1cb5644640 1 -- 172.21.15.103:0/1038018705 wait complete. 2024-09-05T02:01:01.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T02:01:00.899+0000 7f81c5414640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T02:01:01.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T02:01:00.899+0000 7f81c5414640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T02:01:01.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T02:01:00.899+0000 7f81c5414640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T02:01:01.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T02:01:00.899+0000 7f81c5414640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T02:01:01.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:01:00 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T02:01:00.899+0000 7f81c5414640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T02:01:01.614 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-05T02:01:01.860 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:02.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:02 smithi103 ceph-mon[30902]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:02.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:02 smithi103 ceph-mon[30902]: pgmap v304: 129 pgs: 129 active+clean; 585 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-09-05T02:01:02.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:02 smithi103 ceph-mon[30902]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-05T02:01:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.388+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/1077627180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb841082e0 msgr2=0x7fcb8410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.388+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/1077627180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb841082e0 0x7fcb8410ec50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009cd0 tx=0x7fcb7c02a8e0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.388+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/1077627180 shutdown_connections 2024-09-05T02:01:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.388+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/1077627180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb841082e0 0x7fcb8410ec50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.388+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/1077627180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb84103760 0x7fcb84107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.388+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/1077627180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84102db0 0x7fcb84103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.388+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/1077627180 >> 172.21.15.103:0/1077627180 conn(0x7fcb840fc7d0 msgr2=0x7fcb840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/1077627180 shutdown_connections 2024-09-05T02:01:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/1077627180 wait complete. 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 Processor -- start 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 -- start start 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb84102db0 0x7fcb84198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84103760 0x7fcb84198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb841082e0 0x7fcb8419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84111f60 con 0x7fcb84102db0 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84111de0 con 0x7fcb841082e0 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb8b7a8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841120e0 con 0x7fcb84103760 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.389+0000 7fcb89544640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb84102db0 0x7fcb84198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb89544640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb84102db0 0x7fcb84198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54874/0 (socket says 172.21.15.103:54874) 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb89544640 1 -- 172.21.15.103:0/408290533 learned_addr learned my addr 172.21.15.103:0/408290533 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb89d45640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb841082e0 0x7fcb8419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb88d43640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84103760 0x7fcb84198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb89544640 1 -- 172.21.15.103:0/408290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84103760 msgr2=0x7fcb84198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb89544640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84103760 0x7fcb84198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb89544640 1 -- 172.21.15.103:0/408290533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb841082e0 msgr2=0x7fcb8419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb89544640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb841082e0 0x7fcb8419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb89544640 1 -- 172.21.15.103:0/408290533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8419d4f0 con 0x7fcb84102db0 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb88d43640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84103760 0x7fcb84198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb89544640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb84102db0 0x7fcb84198540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400d950 tx=0x7fcb7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb7a7fc640 1 -- 172.21.15.103:0/408290533 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74017070 con 0x7fcb84102db0 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8419d7e0 con 0x7fcb84102db0 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb7a7fc640 1 -- 172.21.15.103:0/408290533 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb74009c30 con 0x7fcb84102db0 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb7a7fc640 1 -- 172.21.15.103:0/408290533 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400f620 con 0x7fcb84102db0 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.390+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb841a50c0 con 0x7fcb84102db0 2024-09-05T02:01:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.392+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb84103190 con 0x7fcb84102db0 2024-09-05T02:01:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.393+0000 7fcb7a7fc640 1 -- 172.21.15.103:0/408290533 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcb740109a0 con 0x7fcb84102db0 2024-09-05T02:01:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.394+0000 7fcb7a7fc640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fcb700a2c10 0x7fcb700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.394+0000 7fcb7a7fc640 1 -- 172.21.15.103:0/408290533 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fcb740cf060 con 0x7fcb84102db0 2024-09-05T02:01:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.394+0000 7fcb88d43640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fcb700a2c10 0x7fcb700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.394+0000 7fcb88d43640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fcb700a2c10 0x7fcb700a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb840fee50 tx=0x7fcb80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.399+0000 7fcb7a7fc640 1 -- 172.21.15.103:0/408290533 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb74009de0 con 0x7fcb84102db0 2024-09-05T02:01:02.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.598+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcb84062630 con 0x7fcb700a2c10 2024-09-05T02:01:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.625+0000 7fcb7a7fc640 1 -- 172.21.15.103:0/408290533 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb84062630 con 0x7fcb700a2c10 2024-09-05T02:01:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.627+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fcb700a2c10 msgr2=0x7fcb700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.627+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fcb700a2c10 0x7fcb700a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb840fee50 tx=0x7fcb80005f90 comp rx=0 tx=0).stop 2024-09-05T02:01:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.627+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb84102db0 msgr2=0x7fcb84198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.627+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb84102db0 0x7fcb84198540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400d950 tx=0x7fcb7400de10 comp rx=0 tx=0).stop 2024-09-05T02:01:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.628+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 shutdown_connections 2024-09-05T02:01:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.628+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb841082e0 0x7fcb8419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.628+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fcb700a2c10 0x7fcb700a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.628+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84103760 0x7fcb84198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.628+0000 7fcb8b7a8640 1 --2- 172.21.15.103:0/408290533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb84102db0 0x7fcb84198540 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.628+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 >> 172.21.15.103:0/408290533 conn(0x7fcb840fc7d0 msgr2=0x7fcb84105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.628+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 shutdown_connections 2024-09-05T02:01:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:02.628+0000 7fcb8b7a8640 1 -- 172.21.15.103:0/408290533 wait complete. 2024-09-05T02:01:02.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:02 smithi167 ceph-mon[35122]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:02.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:02 smithi167 ceph-mon[35122]: pgmap v304: 129 pgs: 129 active+clean; 585 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-09-05T02:01:02.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:02 smithi167 ceph-mon[35122]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-05T02:01:02.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:02 smithi139 ceph-mon[34262]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:02.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:02 smithi139 ceph-mon[34262]: pgmap v304: 129 pgs: 129 active+clean; 585 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-09-05T02:01:02.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:02 smithi139 ceph-mon[34262]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-05T02:01:03.342 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-05T02:01:03.594 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:04.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 -- 172.21.15.103:0/2775212700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c01036e0 msgr2=0x7f71c0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/2775212700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c01036e0 0x7f71c0107b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009930 tx=0x7f71b002a420 comp rx=0 tx=0).stop 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 -- 172.21.15.103:0/2775212700 shutdown_connections 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/2775212700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c01082a0 0x7f71c010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/2775212700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c01036e0 0x7f71c0107b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/2775212700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71c0102d30 0x7f71c0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 -- 172.21.15.103:0/2775212700 >> 172.21.15.103:0/2775212700 conn(0x7f71c00fc7d0 msgr2=0x7f71c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 -- 172.21.15.103:0/2775212700 shutdown_connections 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 -- 172.21.15.103:0/2775212700 wait complete. 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 Processor -- start 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.127+0000 7f71c6c3e640 1 -- start start 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c6c3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0102d30 0x7f71c0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c6c3e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71c01036e0 0x7f71c0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c6c3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c01082a0 0x7f71c019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c6c3e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0111f50 con 0x7f71c0102d30 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c6c3e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0111dd0 con 0x7f71c01082a0 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c6c3e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01120d0 con 0x7f71c01036e0 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c49da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0102d30 0x7f71c0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c49da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0102d30 0x7f71c0198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54884/0 (socket says 172.21.15.103:54884) 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c49da640 1 -- 172.21.15.103:0/67838044 learned_addr learned my addr 172.21.15.103:0/67838044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71b7fff640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71c01036e0 0x7f71c0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c51db640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c01082a0 0x7f71c019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c49da640 1 -- 172.21.15.103:0/67838044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71c01036e0 msgr2=0x7f71c0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c49da640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71c01036e0 0x7f71c0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c49da640 1 -- 172.21.15.103:0/67838044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c01082a0 msgr2=0x7f71c019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:04.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c49da640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c01082a0 0x7f71c019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c49da640 1 -- 172.21.15.103:0/67838044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c019d530 con 0x7f71c0102d30 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71b7fff640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71c01036e0 0x7f71c0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c51db640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c01082a0 0x7f71c019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c49da640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0102d30 0x7f71c0198580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00d770 tx=0x7f71bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71b5ffb640 1 -- 172.21.15.103:0/67838044 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc017070 con 0x7f71c0102d30 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c019d820 con 0x7f71c0102d30 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71b5ffb640 1 -- 172.21.15.103:0/67838044 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71bc0054d0 con 0x7f71c0102d30 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.128+0000 7f71b5ffb640 1 -- 172.21.15.103:0/67838044 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc004cf0 con 0x7f71c0102d30 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.129+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01a5100 con 0x7f71c0102d30 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.130+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c0103110 con 0x7f71c0102d30 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.130+0000 7f71b5ffb640 1 -- 172.21.15.103:0/67838044 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f71bc004e90 con 0x7f71c0102d30 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.131+0000 7f71b5ffb640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f71a80a2c10 0x7f71a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.131+0000 7f71b5ffb640 1 -- 172.21.15.103:0/67838044 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f71bc0cef30 con 0x7f71c0102d30 2024-09-05T02:01:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.131+0000 7f71b7fff640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f71a80a2c10 0x7f71a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:04.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.132+0000 7f71b7fff640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f71a80a2c10 0x7f71a80a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009900 tx=0x7f71b0032c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:04.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.135+0000 7f71b5ffb640 1 -- 172.21.15.103:0/67838044 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71bc09baf0 con 0x7f71c0102d30 2024-09-05T02:01:04.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.330+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f71c0062630 con 0x7f71a80a2c10 2024-09-05T02:01:04.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:04 smithi103 ceph-mon[30902]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:04.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:04 smithi103 ceph-mon[30902]: pgmap v305: 129 pgs: 129 active+clean; 586 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-05T02:01:04.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.335+0000 7f71b5ffb640 1 -- 172.21.15.103:0/67838044 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f71c0062630 con 0x7f71a80a2c10 2024-09-05T02:01:04.336 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_0/326867bd-ec7c-4ddf-a493-aa41608a5e39 2024-09-05T02:01:04.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.336+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f71a80a2c10 msgr2=0x7f71a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:04.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.336+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f71a80a2c10 0x7f71a80a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009900 tx=0x7f71b0032c20 comp rx=0 tx=0).stop 2024-09-05T02:01:04.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.336+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0102d30 msgr2=0x7f71c0198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:04.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.336+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0102d30 0x7f71c0198580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00d770 tx=0x7f71bc00dc30 comp rx=0 tx=0).stop 2024-09-05T02:01:04.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.337+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 shutdown_connections 2024-09-05T02:01:04.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.337+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c01082a0 0x7f71c019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:04.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.337+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f71a80a2c10 0x7f71a80a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:04.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.337+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71c01036e0 0x7f71c0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:04.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.337+0000 7f71c6c3e640 1 --2- 172.21.15.103:0/67838044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0102d30 0x7f71c0198580 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:04.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.337+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 >> 172.21.15.103:0/67838044 conn(0x7f71c00fc7d0 msgr2=0x7f71c0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:04.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.337+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 shutdown_connections 2024-09-05T02:01:04.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:04.337+0000 7f71c6c3e640 1 -- 172.21.15.103:0/67838044 wait complete. 2024-09-05T02:01:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:04 smithi167 ceph-mon[35122]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:04 smithi167 ceph-mon[35122]: pgmap v305: 129 pgs: 129 active+clean; 586 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-05T02:01:04.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:04 smithi139 ceph-mon[34262]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:04.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:04 smithi139 ceph-mon[34262]: pgmap v305: 129 pgs: 129 active+clean; 586 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-05T02:01:04.978 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-05T02:01:05.225 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.752+0000 7f99e296a640 1 -- 172.21.15.103:0/1018842530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc106f10 msgr2=0x7f99dc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.752+0000 7f99e296a640 1 --2- 172.21.15.103:0/1018842530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc106f10 0x7f99dc10ea90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009ca0 tx=0x7f99d402a8c0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.752+0000 7f99e296a640 1 -- 172.21.15.103:0/1018842530 shutdown_connections 2024-09-05T02:01:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.752+0000 7f99e296a640 1 --2- 172.21.15.103:0/1018842530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc106f10 0x7f99dc10ea90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.752+0000 7f99e296a640 1 --2- 172.21.15.103:0/1018842530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99dc102790 0x7f99dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.752+0000 7f99e296a640 1 --2- 172.21.15.103:0/1018842530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99dc101de0 0x7f99dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.752+0000 7f99e296a640 1 -- 172.21.15.103:0/1018842530 >> 172.21.15.103:0/1018842530 conn(0x7f99dc0fd8c0 msgr2=0x7f99dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.752+0000 7f99e296a640 1 -- 172.21.15.103:0/1018842530 shutdown_connections 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.752+0000 7f99e296a640 1 -- 172.21.15.103:0/1018842530 wait complete. 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.753+0000 7f99e296a640 1 Processor -- start 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.753+0000 7f99e296a640 1 -- start start 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.753+0000 7f99e296a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc101de0 0x7f99dc1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.753+0000 7f99e296a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99dc102790 0x7f99dc198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.753+0000 7f99e296a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99dc106f10 0x7f99dc19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.753+0000 7f99e296a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc111e70 con 0x7f99dc101de0 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.753+0000 7f99e296a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc111cf0 con 0x7f99dc106f10 2024-09-05T02:01:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.753+0000 7f99e296a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc111ff0 con 0x7f99dc102790 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.753+0000 7f99dbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc101de0 0x7f99dc1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99dbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc101de0 0x7f99dc1982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54912/0 (socket says 172.21.15.103:54912) 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99dbfff640 1 -- 172.21.15.103:0/839768091 learned_addr learned my addr 172.21.15.103:0/839768091 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99db7fe640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99dc102790 0x7f99dc198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99e0f07640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99dc106f10 0x7f99dc19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99dbfff640 1 -- 172.21.15.103:0/839768091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99dc102790 msgr2=0x7f99dc198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99dbfff640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99dc102790 0x7f99dc198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99dbfff640 1 -- 172.21.15.103:0/839768091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99dc106f10 msgr2=0x7f99dc19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99dbfff640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99dc106f10 0x7f99dc19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99dbfff640 1 -- 172.21.15.103:0/839768091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc19d2a0 con 0x7f99dc101de0 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99e0f07640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99dc106f10 0x7f99dc19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99dbfff640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc101de0 0x7f99dc1982f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d770 tx=0x7f99cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99d97fa640 1 -- 172.21.15.103:0/839768091 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc017070 con 0x7f99dc101de0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc19d590 con 0x7f99dc101de0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99d97fa640 1 -- 172.21.15.103:0/839768091 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99cc0054d0 con 0x7f99dc101de0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99d97fa640 1 -- 172.21.15.103:0/839768091 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc004cf0 con 0x7f99dc101de0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.754+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc1a4e70 con 0x7f99dc101de0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.755+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99dc1021c0 con 0x7f99dc101de0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.756+0000 7f99d97fa640 1 -- 172.21.15.103:0/839768091 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f99cc004e90 con 0x7f99dc101de0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.757+0000 7f99d97fa640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99c40a2c10 0x7f99c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.757+0000 7f99db7fe640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99c40a2c10 0x7f99c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.757+0000 7f99d97fa640 1 -- 172.21.15.103:0/839768091 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f99cc0cee80 con 0x7f99dc101de0 2024-09-05T02:01:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.757+0000 7f99db7fe640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99c40a2c10 0x7f99c40a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99dc108880 tx=0x7f99d000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:05.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.761+0000 7f99d97fa640 1 -- 172.21.15.103:0/839768091 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99cc09ba40 con 0x7f99dc101de0 2024-09-05T02:01:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.953+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f99dc10a160 con 0x7f99c40a2c10 2024-09-05T02:01:05.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.978+0000 7f99d97fa640 1 -- 172.21.15.103:0/839768091 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f99d0010b70 con 0x7f99c40a2c10 2024-09-05T02:01:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99c40a2c10 msgr2=0x7f99c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99c40a2c10 0x7f99c40a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99dc108880 tx=0x7f99d000b390 comp rx=0 tx=0).stop 2024-09-05T02:01:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc101de0 msgr2=0x7f99dc1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc101de0 0x7f99dc1982f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d770 tx=0x7f99cc00dc30 comp rx=0 tx=0).stop 2024-09-05T02:01:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 shutdown_connections 2024-09-05T02:01:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99dc106f10 0x7f99dc19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99c40a2c10 0x7f99c40a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99dc102790 0x7f99dc198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 --2- 172.21.15.103:0/839768091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99dc101de0 0x7f99dc1982f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:05.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 >> 172.21.15.103:0/839768091 conn(0x7f99dc0fd8c0 msgr2=0x7f99dc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:05.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 shutdown_connections 2024-09-05T02:01:05.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:05.980+0000 7f99e296a640 1 -- 172.21.15.103:0/839768091 wait complete. 2024-09-05T02:01:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:06 smithi167 ceph-mon[35122]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:06 smithi167 ceph-mon[35122]: pgmap v306: 129 pgs: 129 active+clean; 612 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-09-05T02:01:06.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:06 smithi139 ceph-mon[34262]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:06.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:06 smithi139 ceph-mon[34262]: pgmap v306: 129 pgs: 129 active+clean; 612 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-09-05T02:01:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:06 smithi103 ceph-mon[30902]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:06 smithi103 ceph-mon[30902]: pgmap v306: 129 pgs: 129 active+clean; 612 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-09-05T02:01:06.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-05T02:01:06.930 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:07.353 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:07 smithi103 ceph-mon[30902]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:07.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.437+0000 7f261e1df640 1 -- 172.21.15.103:0/1130730377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 msgr2=0x7f2618100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:07.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.437+0000 7f261e1df640 1 --2- 172.21.15.103:0/1130730377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 0x7f2618100ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f260c009930 tx=0x7f260c02a420 comp rx=0 tx=0).stop 2024-09-05T02:01:07.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.437+0000 7f261e1df640 1 -- 172.21.15.103:0/1130730377 shutdown_connections 2024-09-05T02:01:07.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.437+0000 7f261e1df640 1 --2- 172.21.15.103:0/1130730377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2618101420 0x7f261810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:07.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.437+0000 7f261e1df640 1 --2- 172.21.15.103:0/1130730377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 0x7f2618100ee0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:07.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.437+0000 7f261e1df640 1 --2- 172.21.15.103:0/1130730377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2618106a80 0x7f2618106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:07.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.437+0000 7f261e1df640 1 -- 172.21.15.103:0/1130730377 >> 172.21.15.103:0/1130730377 conn(0x7f26180fc7a0 msgr2=0x7f26180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.437+0000 7f261e1df640 1 -- 172.21.15.103:0/1130730377 shutdown_connections 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.437+0000 7f261e1df640 1 -- 172.21.15.103:0/1130730377 wait complete. 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f261e1df640 1 Processor -- start 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f261e1df640 1 -- start start 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f261e1df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 0x7f26181983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f261e1df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2618101420 0x7f2618198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f261e1df640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2618106a80 0x7f261819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f261e1df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2618110d30 con 0x7f2618100a80 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f261e1df640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2618110bb0 con 0x7f2618101420 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f261e1df640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2618110eb0 con 0x7f2618106a80 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f26177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 0x7f26181983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f26177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 0x7f26181983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54924/0 (socket says 172.21.15.103:54924) 2024-09-05T02:01:07.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f26177fe640 1 -- 172.21.15.103:0/2781344498 learned_addr learned my addr 172.21.15.103:0/2781344498 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f26177fe640 1 -- 172.21.15.103:0/2781344498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2618106a80 msgr2=0x7f261819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f26177fe640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2618106a80 0x7f261819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f26177fe640 1 -- 172.21.15.103:0/2781344498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2618101420 msgr2=0x7f2618198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f2616ffd640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2618101420 0x7f2618198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f26177fe640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2618101420 0x7f2618198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f26177fe640 1 -- 172.21.15.103:0/2781344498 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261819d370 con 0x7f2618100a80 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.438+0000 7f2616ffd640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2618101420 0x7f2618198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.439+0000 7f26177fe640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 0x7f26181983c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f260800d770 tx=0x7f260800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.439+0000 7f2614ff9640 1 -- 172.21.15.103:0/2781344498 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608017070 con 0x7f2618100a80 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.439+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261819d660 con 0x7f2618100a80 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.439+0000 7f2614ff9640 1 -- 172.21.15.103:0/2781344498 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26080054d0 con 0x7f2618100a80 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.439+0000 7f2614ff9640 1 -- 172.21.15.103:0/2781344498 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608004cf0 con 0x7f2618100a80 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.439+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26181a4f40 con 0x7f2618100a80 2024-09-05T02:01:07.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.440+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2618100ee0 con 0x7f2618100a80 2024-09-05T02:01:07.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.441+0000 7f2614ff9640 1 -- 172.21.15.103:0/2781344498 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2608004e90 con 0x7f2618100a80 2024-09-05T02:01:07.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.442+0000 7f2614ff9640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f26000a2c10 0x7f26000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:07.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.442+0000 7f2616ffd640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f26000a2c10 0x7f26000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:07.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.442+0000 7f2614ff9640 1 -- 172.21.15.103:0/2781344498 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f26080cf230 con 0x7f2618100a80 2024-09-05T02:01:07.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.442+0000 7f2616ffd640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f26000a2c10 0x7f26000a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c00 tx=0x7f260c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:07.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.445+0000 7f2614ff9640 1 -- 172.21.15.103:0/2781344498 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f260809be90 con 0x7f2618100a80 2024-09-05T02:01:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.633+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f2618062630 con 0x7f26000a2c10 2024-09-05T02:01:07.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:07 smithi167 ceph-mon[35122]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.638+0000 7f2614ff9640 1 -- 172.21.15.103:0/2781344498 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2618062630 con 0x7f26000a2c10 2024-09-05T02:01:07.639 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_1/d766642e-a15e-4591-a6fc-f00cde35e370 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f26000a2c10 msgr2=0x7f26000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f26000a2c10 0x7f26000a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c00 tx=0x7f260c031400 comp rx=0 tx=0).stop 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 msgr2=0x7f26181983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 0x7f26181983c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f260800d770 tx=0x7f260800dc30 comp rx=0 tx=0).stop 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 shutdown_connections 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f26000a2c10 0x7f26000a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2618106a80 0x7f261819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2618101420 0x7f2618198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 --2- 172.21.15.103:0/2781344498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100a80 0x7f26181983c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 >> 172.21.15.103:0/2781344498 conn(0x7f26180fc7a0 msgr2=0x7f261806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 shutdown_connections 2024-09-05T02:01:07.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:07.640+0000 7f261e1df640 1 -- 172.21.15.103:0/2781344498 wait complete. 2024-09-05T02:01:07.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:07 smithi139 ceph-mon[34262]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:08.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:01:08.609 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:08 smithi167 ceph-mon[35122]: pgmap v307: 129 pgs: 129 active+clean; 612 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-09-05T02:01:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:08 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:01:08.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:08 smithi139 ceph-mon[34262]: pgmap v307: 129 pgs: 129 active+clean; 612 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-09-05T02:01:08.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:08 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:01:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:08 smithi103 ceph-mon[30902]: pgmap v307: 129 pgs: 129 active+clean; 612 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-09-05T02:01:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:08 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:01:09.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.311+0000 7f13bd75f640 1 -- 172.21.15.103:0/3348930786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8106f10 msgr2=0x7f13b810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:09.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.311+0000 7f13bd75f640 1 --2- 172.21.15.103:0/3348930786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8106f10 0x7f13b810ea90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009ca0 tx=0x7f13b002a8c0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.311+0000 7f13bd75f640 1 -- 172.21.15.103:0/3348930786 shutdown_connections 2024-09-05T02:01:09.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.311+0000 7f13bd75f640 1 --2- 172.21.15.103:0/3348930786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8106f10 0x7f13b810ea90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.311+0000 7f13bd75f640 1 --2- 172.21.15.103:0/3348930786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8102790 0x7f13b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.311+0000 7f13bd75f640 1 --2- 172.21.15.103:0/3348930786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13b8101de0 0x7f13b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.311+0000 7f13bd75f640 1 -- 172.21.15.103:0/3348930786 >> 172.21.15.103:0/3348930786 conn(0x7f13b80fd8c0 msgr2=0x7f13b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.311+0000 7f13bd75f640 1 -- 172.21.15.103:0/3348930786 shutdown_connections 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.311+0000 7f13bd75f640 1 -- 172.21.15.103:0/3348930786 wait complete. 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13bd75f640 1 Processor -- start 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13bd75f640 1 -- start start 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13bd75f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13b8101de0 0x7f13b8198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13bd75f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8102790 0x7f13b8198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:09.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13bd75f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8106f10 0x7f13b819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13bd75f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8111e20 con 0x7f13b8102790 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13bd75f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8111ca0 con 0x7f13b8106f10 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13bd75f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8111fa0 con 0x7f13b8101de0 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8102790 0x7f13b8198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8102790 0x7f13b8198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41686/0 (socket says 172.21.15.103:41686) 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b67fc640 1 -- 172.21.15.103:0/1889026648 learned_addr learned my addr 172.21.15.103:0/1889026648 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b6ffd640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13b8101de0 0x7f13b8198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b77fe640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8106f10 0x7f13b819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b67fc640 1 -- 172.21.15.103:0/1889026648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13b8101de0 msgr2=0x7f13b8198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b67fc640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13b8101de0 0x7f13b8198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b67fc640 1 -- 172.21.15.103:0/1889026648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8106f10 msgr2=0x7f13b819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b67fc640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8106f10 0x7f13b819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.312+0000 7f13b67fc640 1 -- 172.21.15.103:0/1889026648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b819d1f0 con 0x7f13b8102790 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.313+0000 7f13b67fc640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8102790 0x7f13b8198780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f13a000d950 tx=0x7f13a000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.313+0000 7f13affff640 1 -- 172.21.15.103:0/1889026648 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0017030 con 0x7f13b8102790 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.313+0000 7f13affff640 1 -- 172.21.15.103:0/1889026648 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13a0004020 con 0x7f13b8102790 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.313+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b819d4e0 con 0x7f13b8102790 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.313+0000 7f13affff640 1 -- 172.21.15.103:0/1889026648 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0005570 con 0x7f13b8102790 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.313+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b81a4dc0 con 0x7f13b8102790 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.314+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b81021c0 con 0x7f13b8102790 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.315+0000 7f13affff640 1 -- 172.21.15.103:0/1889026648 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f13a00041d0 con 0x7f13b8102790 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.315+0000 7f13affff640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f13a40a2c10 0x7f13a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.316+0000 7f13b6ffd640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f13a40a2c10 0x7f13a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.316+0000 7f13affff640 1 -- 172.21.15.103:0/1889026648 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f13a00cecb0 con 0x7f13b8102790 2024-09-05T02:01:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.316+0000 7f13b6ffd640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f13a40a2c10 0x7f13a40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f13a800abe0 tx=0x7f13a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.320+0000 7f13affff640 1 -- 172.21.15.103:0/1889026648 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13a009b870 con 0x7f13b8102790 2024-09-05T02:01:09.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:09 smithi103 ceph-mon[30902]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:09.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.527+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13b8109b60 con 0x7f13b8102790 2024-09-05T02:01:09.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.528+0000 7f13affff640 1 -- 172.21.15.103:0/1889026648 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7f13a009b170 con 0x7f13b8102790 2024-09-05T02:01:09.530 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:09.530 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":28,"btime":"2024-09-05T02:00:59:232233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:00:59.232218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:01:09.530 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 28 2024-09-05T02:01:09.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f13a40a2c10 msgr2=0x7f13a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:09.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f13a40a2c10 0x7f13a40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f13a800abe0 tx=0x7f13a800a2d0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8102790 msgr2=0x7f13b8198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:09.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8102790 0x7f13b8198780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f13a000d950 tx=0x7f13a000dd50 comp rx=0 tx=0).stop 2024-09-05T02:01:09.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 shutdown_connections 2024-09-05T02:01:09.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8106f10 0x7f13b819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13b8102790 0x7f13b8198780 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f13a40a2c10 0x7f13a40a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 --2- 172.21.15.103:0/1889026648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13b8101de0 0x7f13b8198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:09.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.530+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 >> 172.21.15.103:0/1889026648 conn(0x7f13b80fd8c0 msgr2=0x7f13b80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:09.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.531+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 shutdown_connections 2024-09-05T02:01:09.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:09.531+0000 7f13bd75f640 1 -- 172.21.15.103:0/1889026648 wait complete. 2024-09-05T02:01:09.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:09 smithi167 ceph-mon[35122]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:09.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:09 smithi139 ceph-mon[34262]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:01:10.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:10 smithi103 ceph-mon[30902]: pgmap v308: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 24 op/s 2024-09-05T02:01:10.507 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:10 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1889026648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:01:10.507 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.507 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.507 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.507 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:10 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:01:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:10 smithi167 ceph-mon[35122]: pgmap v308: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 24 op/s 2024-09-05T02:01:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:10 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1889026648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:01:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:10 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:10 smithi139 ceph-mon[34262]: pgmap v308: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 24 op/s 2024-09-05T02:01:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:10 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1889026648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:01:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:10 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:10.817 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:11.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.375+0000 7efdfbb13640 1 -- 172.21.15.103:0/2029231798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4105f40 msgr2=0x7efdf41063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:11.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.375+0000 7efdfbb13640 1 --2- 172.21.15.103:0/2029231798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4105f40 0x7efdf41063d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efdec009ca0 tx=0x7efdec02a940 comp rx=0 tx=0).stop 2024-09-05T02:01:11.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.375+0000 7efdfbb13640 1 -- 172.21.15.103:0/2029231798 shutdown_connections 2024-09-05T02:01:11.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.375+0000 7efdfbb13640 1 --2- 172.21.15.103:0/2029231798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4105f40 0x7efdf41063d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.375+0000 7efdfbb13640 1 --2- 172.21.15.103:0/2029231798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdf4101450 0x7efdf4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.375+0000 7efdfbb13640 1 --2- 172.21.15.103:0/2029231798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf4100b30 0x7efdf4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.375+0000 7efdfbb13640 1 -- 172.21.15.103:0/2029231798 >> 172.21.15.103:0/2029231798 conn(0x7efdf40fc7a0 msgr2=0x7efdf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 -- 172.21.15.103:0/2029231798 shutdown_connections 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 -- 172.21.15.103:0/2029231798 wait complete. 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 Processor -- start 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 -- start start 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4100b30 0x7efdf41a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdf4101450 0x7efdf41a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf4105f40 0x7efdf419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efdf4118420 con 0x7efdf4100b30 2024-09-05T02:01:11.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41182a0 con 0x7efdf4101450 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdfbb13640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41185a0 con 0x7efdf4105f40 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdf98af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4100b30 0x7efdf41a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdf98af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4100b30 0x7efdf41a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41700/0 (socket says 172.21.15.103:41700) 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.376+0000 7efdf98af640 1 -- 172.21.15.103:0/482191771 learned_addr learned my addr 172.21.15.103:0/482191771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdf90ae640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdf4101450 0x7efdf41a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdf98af640 1 -- 172.21.15.103:0/482191771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf4105f40 msgr2=0x7efdf419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdfa0b0640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf4105f40 0x7efdf419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdf98af640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf4105f40 0x7efdf419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdf98af640 1 -- 172.21.15.103:0/482191771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdf4101450 msgr2=0x7efdf41a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdf98af640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdf4101450 0x7efdf41a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdf98af640 1 -- 172.21.15.103:0/482191771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf419ef40 con 0x7efdf4100b30 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdf98af640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4100b30 0x7efdf41a4600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efdf000d770 tx=0x7efdf000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdeaffd640 1 -- 172.21.15.103:0/482191771 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0017070 con 0x7efdf4100b30 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdeaffd640 1 -- 172.21.15.103:0/482191771 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf00054d0 con 0x7efdf4100b30 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf419f230 con 0x7efdf4100b30 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdeaffd640 1 -- 172.21.15.103:0/482191771 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0004cf0 con 0x7efdf4100b30 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.377+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf41ab3b0 con 0x7efdf4100b30 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.379+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf406b070 con 0x7efdf4100b30 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.379+0000 7efdeaffd640 1 -- 172.21.15.103:0/482191771 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efdf0004e90 con 0x7efdf4100b30 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.380+0000 7efdeaffd640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efde00a2c10 0x7efde00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.380+0000 7efdf90ae640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efde00a2c10 0x7efde00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.380+0000 7efdeaffd640 1 -- 172.21.15.103:0/482191771 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7efdf00cef90 con 0x7efdf4100b30 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.381+0000 7efdf90ae640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efde00a2c10 0x7efde00a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efde4005fd0 tx=0x7efde4005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.384+0000 7efdeaffd640 1 -- 172.21.15.103:0/482191771 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdf009baf0 con 0x7efdf4100b30 2024-09-05T02:01:11.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.555+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdf41ab580 con 0x7efdf4100b30 2024-09-05T02:01:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.556+0000 7efdeaffd640 1 -- 172.21.15.103:0/482191771 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7efdf009b3f0 con 0x7efdf4100b30 2024-09-05T02:01:11.558 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:11.558 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:00:24.085875+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:01:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efde00a2c10 msgr2=0x7efde00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efde00a2c10 0x7efde00a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efde4005fd0 tx=0x7efde4005f60 comp rx=0 tx=0).stop 2024-09-05T02:01:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4100b30 msgr2=0x7efdf41a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4100b30 0x7efdf41a4600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efdf000d770 tx=0x7efdf000dc30 comp rx=0 tx=0).stop 2024-09-05T02:01:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 shutdown_connections 2024-09-05T02:01:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efde00a2c10 0x7efde00a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdf4105f40 0x7efdf419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdf4101450 0x7efdf41a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 --2- 172.21.15.103:0/482191771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdf4100b30 0x7efdf41a4600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.558+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 >> 172.21.15.103:0/482191771 conn(0x7efdf40fc7a0 msgr2=0x7efdf41038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.559+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 shutdown_connections 2024-09-05T02:01:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:11.559+0000 7efdfbb13640 1 -- 172.21.15.103:0/482191771 wait complete. 2024-09-05T02:01:12.116 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:12.116 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:12.116 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:11 smithi103 ceph-mon[30902]: pgmap v309: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 21 op/s 2024-09-05T02:01:12.116 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:01:12.116 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:01:12.117 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:11 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:12.117 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:11 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/482191771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:01:12.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:12.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:12.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:11 smithi167 ceph-mon[35122]: pgmap v309: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 21 op/s 2024-09-05T02:01:12.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:01:12.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:01:12.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:11 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:12.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:11 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/482191771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:01:12.143 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T02:01:12.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T02:01:12.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:12.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:12.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:11 smithi139 ceph-mon[34262]: pgmap v309: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 21 op/s 2024-09-05T02:01:12.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:01:12.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:01:12.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:11 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:01:12.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:11 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/482191771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:01:12.391 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:12.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.840+0000 7f9049b0c640 1 -- 172.21.15.103:0/725080485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 msgr2=0x7f904410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:12.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.840+0000 7f9049b0c640 1 --2- 172.21.15.103:0/725080485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 0x7f904410d950 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f903c009cd0 tx=0x7f903c02a8e0 comp rx=0 tx=0).stop 2024-09-05T02:01:12.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.840+0000 7f9049b0c640 1 -- 172.21.15.103:0/725080485 shutdown_connections 2024-09-05T02:01:12.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.840+0000 7f9049b0c640 1 --2- 172.21.15.103:0/725080485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 0x7f904410d950 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:12.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.840+0000 7f9049b0c640 1 --2- 172.21.15.103:0/725080485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9044100a80 0x7f9044100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:12.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.840+0000 7f9049b0c640 1 --2- 172.21.15.103:0/725080485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9044106a80 0x7f9044106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:12.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.840+0000 7f9049b0c640 1 -- 172.21.15.103:0/725080485 >> 172.21.15.103:0/725080485 conn(0x7f90440fc7a0 msgr2=0x7f90440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:12.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.840+0000 7f9049b0c640 1 -- 172.21.15.103:0/725080485 shutdown_connections 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.840+0000 7f9049b0c640 1 -- 172.21.15.103:0/725080485 wait complete. 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9049b0c640 1 Processor -- start 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9049b0c640 1 -- start start 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9049b0c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9044100a80 0x7f90441991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9049b0c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 0x7f9044199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9049b0c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9044106a80 0x7f90441a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9049b0c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9044110d30 con 0x7f9044101420 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9049b0c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9044110bb0 con 0x7f9044100a80 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9049b0c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9044110eb0 con 0x7f9044106a80 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9042ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 0x7f9044199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9042ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 0x7f9044199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41714/0 (socket says 172.21.15.103:41714) 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9042ffd640 1 -- 172.21.15.103:0/1423538505 learned_addr learned my addr 172.21.15.103:0/1423538505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:12.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f90437fe640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9044100a80 0x7f90441991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.841+0000 7f9043fff640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9044106a80 0x7f90441a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9042ffd640 1 -- 172.21.15.103:0/1423538505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9044106a80 msgr2=0x7f90441a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9042ffd640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9044106a80 0x7f90441a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9042ffd640 1 -- 172.21.15.103:0/1423538505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9044100a80 msgr2=0x7f90441991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9042ffd640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9044100a80 0x7f90441991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9042ffd640 1 -- 172.21.15.103:0/1423538505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90441a4780 con 0x7f9044101420 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9042ffd640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 0x7f9044199700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f903800d950 tx=0x7f903800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9040ff9640 1 -- 172.21.15.103:0/1423538505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038017070 con 0x7f9044101420 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9040ff9640 1 -- 172.21.15.103:0/1423538505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9038009c30 con 0x7f9044101420 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9040ff9640 1 -- 172.21.15.103:0/1423538505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903800f620 con 0x7f9044101420 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90441a4950 con 0x7f9044101420 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.842+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90441a4e90 con 0x7f9044101420 2024-09-05T02:01:12.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.843+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9044100ee0 con 0x7f9044101420 2024-09-05T02:01:12.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.844+0000 7f9040ff9640 1 -- 172.21.15.103:0/1423538505 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f90380109a0 con 0x7f9044101420 2024-09-05T02:01:12.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.845+0000 7f9040ff9640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f902c0a2c90 0x7f902c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:12.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.845+0000 7f90437fe640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f902c0a2c90 0x7f902c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:12.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.845+0000 7f9040ff9640 1 -- 172.21.15.103:0/1423538505 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f90380cef30 con 0x7f9044101420 2024-09-05T02:01:12.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.845+0000 7f90437fe640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f902c0a2c90 0x7f902c0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9034005fd0 tx=0x7f903400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:12.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:12.849+0000 7f9040ff9640 1 -- 172.21.15.103:0/1423538505 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9038009de0 con 0x7f9044101420 2024-09-05T02:01:13.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.017+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9044109080 con 0x7f902c0a2c90 2024-09-05T02:01:13.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.026+0000 7f9040ff9640 1 -- 172.21.15.103:0/1423538505 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340379 (secure 0 0 0) 0x7f9034003870 con 0x7f902c0a2c90 2024-09-05T02:01:13.031 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:13.033 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-05T02:01:13.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f902c0a2c90 msgr2=0x7f902c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:13.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f902c0a2c90 0x7f902c0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9034005fd0 tx=0x7f903400b390 comp rx=0 tx=0).stop 2024-09-05T02:01:13.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 msgr2=0x7f9044199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:13.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 0x7f9044199700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f903800d950 tx=0x7f903800de10 comp rx=0 tx=0).stop 2024-09-05T02:01:13.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 shutdown_connections 2024-09-05T02:01:13.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f902c0a2c90 0x7f902c0a5150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:13.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9044106a80 0x7f90441a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:13.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9044101420 0x7f9044199700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:13.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 --2- 172.21.15.103:0/1423538505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9044100a80 0x7f90441991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:13.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 >> 172.21.15.103:0/1423538505 conn(0x7f90440fc7a0 msgr2=0x7f90440fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:13.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 shutdown_connections 2024-09-05T02:01:13.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:13.029+0000 7f9049b0c640 1 -- 172.21.15.103:0/1423538505 wait complete. 2024-09-05T02:01:13.695 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-05T02:01:11.094948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650218,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":264,"num_read_kb":236,"num_write":421,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391892,"kb_used_data":13524,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549932,"statfs":{"total":1151940427776,"available":1151539130368,"internally_reserved":0,"allocated":13848576,"data_stored":7650257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54692,"num_objects":49,"num_object_clones":0,"num_object_copies":149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":138,"num_read_kb":127,"num_write":124,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002888"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:07.252880+0000","last_change":"2024-09-05T02:00:12.347638+0000","last_active":"2024-09-05T02:01:07.252880+0000","last_peered":"2024-09-05T02:01:07.252880+0000","last_clean":"2024-09-05T02:01:07.252880+0000","last_became_active":"2024-09-05T02:00:12.347314+0000","last_became_peered":"2024-09-05T02:00:12.347314+0000","last_unstale":"2024-09-05T02:01:07.252880+0000","last_undegraded":"2024-09-05T02:01:07.252880+0000","last_fullsized":"2024-09-05T02:01:07.252880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:08:39.084117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416951+0000","last_change":"2024-09-05T02:00:14.360262+0000","last_active":"2024-09-05T02:00:24.416951+0000","last_peered":"2024-09-05T02:00:24.416951+0000","last_clean":"2024-09-05T02:00:24.416951+0000","last_became_active":"2024-09-05T02:00:14.359965+0000","last_became_peered":"2024-09-05T02:00:14.359965+0000","last_unstale":"2024-09-05T02:00:24.416951+0000","last_undegraded":"2024-09-05T02:00:24.416951+0000","last_fullsized":"2024-09-05T02:00:24.416951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:01:56.109169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416298+0000","last_change":"2024-09-05T02:00:21.410692+0000","last_active":"2024-09-05T02:00:24.416298+0000","last_peered":"2024-09-05T02:00:24.416298+0000","last_clean":"2024-09-05T02:00:24.416298+0000","last_became_active":"2024-09-05T02:00:21.410298+0000","last_became_peered":"2024-09-05T02:00:21.410298+0000","last_unstale":"2024-09-05T02:00:24.416298+0000","last_undegraded":"2024-09-05T02:00:24.416298+0000","last_fullsized":"2024-09-05T02:00:24.416298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:25:31.241700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.417170+0000","last_change":"2024-09-05T02:00:12.783109+0000","last_active":"2024-09-05T02:00:24.417170+0000","last_peered":"2024-09-05T02:00:24.417170+0000","last_clean":"2024-09-05T02:00:24.417170+0000","last_became_active":"2024-09-05T02:00:12.782773+0000","last_became_peered":"2024-09-05T02:00:12.782773+0000","last_unstale":"2024-09-05T02:00:24.417170+0000","last_undegraded":"2024-09-05T02:00:24.417170+0000","last_fullsized":"2024-09-05T02:00:24.417170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:29:12.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421317+0000","last_change":"2024-09-05T02:00:14.370425+0000","last_active":"2024-09-05T02:00:25.421317+0000","last_peered":"2024-09-05T02:00:25.421317+0000","last_clean":"2024-09-05T02:00:25.421317+0000","last_became_active":"2024-09-05T02:00:14.370102+0000","last_became_peered":"2024-09-05T02:00:14.370102+0000","last_unstale":"2024-09-05T02:00:25.421317+0000","last_undegraded":"2024-09-05T02:00:25.421317+0000","last_fullsized":"2024-09-05T02:00:25.421317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:22:43.837418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422396+0000","last_change":"2024-09-05T02:00:21.413082+0000","last_active":"2024-09-05T02:00:25.422396+0000","last_peered":"2024-09-05T02:00:25.422396+0000","last_clean":"2024-09-05T02:00:25.422396+0000","last_became_active":"2024-09-05T02:00:21.412661+0000","last_became_peered":"2024-09-05T02:00:21.412661+0000","last_unstale":"2024-09-05T02:00:25.422396+0000","last_undegraded":"2024-09-05T02:00:25.422396+0000","last_fullsized":"2024-09-05T02:00:25.422396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:51:41.225573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:59.240594+0000","last_change":"2024-09-05T02:00:12.353894+0000","last_active":"2024-09-05T02:00:59.240594+0000","last_peered":"2024-09-05T02:00:59.240594+0000","last_clean":"2024-09-05T02:00:59.240594+0000","last_became_active":"2024-09-05T02:00:12.353483+0000","last_became_peered":"2024-09-05T02:00:12.353483+0000","last_unstale":"2024-09-05T02:00:59.240594+0000","last_undegraded":"2024-09-05T02:00:59.240594+0000","last_fullsized":"2024-09-05T02:00:59.240594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:42.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421480+0000","last_change":"2024-09-05T02:00:14.356441+0000","last_active":"2024-09-05T02:00:25.421480+0000","last_peered":"2024-09-05T02:00:25.421480+0000","last_clean":"2024-09-05T02:00:25.421480+0000","last_became_active":"2024-09-05T02:00:14.356092+0000","last_became_peered":"2024-09-05T02:00:14.356092+0000","last_unstale":"2024-09-05T02:00:25.421480+0000","last_undegraded":"2024-09-05T02:00:25.421480+0000","last_fullsized":"2024-09-05T02:00:25.421480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:33:55.416968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423573+0000","last_change":"2024-09-05T02:00:21.418324+0000","last_active":"2024-09-05T02:00:25.423573+0000","last_peered":"2024-09-05T02:00:25.423573+0000","last_clean":"2024-09-05T02:00:25.423573+0000","last_became_active":"2024-09-05T02:00:21.417934+0000","last_became_peered":"2024-09-05T02:00:21.417934+0000","last_unstale":"2024-09-05T02:00:25.423573+0000","last_undegraded":"2024-09-05T02:00:25.423573+0000","last_fullsized":"2024-09-05T02:00:25.423573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:50.425823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.230762+0000","last_change":"2024-09-05T02:00:12.348637+0000","last_active":"2024-09-05T02:00:58.230762+0000","last_peered":"2024-09-05T02:00:58.230762+0000","last_clean":"2024-09-05T02:00:58.230762+0000","last_became_active":"2024-09-05T02:00:12.348287+0000","last_became_peered":"2024-09-05T02:00:12.348287+0000","last_unstale":"2024-09-05T02:00:58.230762+0000","last_undegraded":"2024-09-05T02:00:58.230762+0000","last_fullsized":"2024-09-05T02:00:58.230762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:52:58.368026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.691044+0000","last_change":"2024-09-05T02:00:14.368414+0000","last_active":"2024-09-05T02:00:24.691044+0000","last_peered":"2024-09-05T02:00:24.691044+0000","last_clean":"2024-09-05T02:00:24.691044+0000","last_became_active":"2024-09-05T02:00:14.368018+0000","last_became_peered":"2024-09-05T02:00:14.368018+0000","last_unstale":"2024-09-05T02:00:24.691044+0000","last_undegraded":"2024-09-05T02:00:24.691044+0000","last_fullsized":"2024-09-05T02:00:24.691044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:08.246420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423072+0000","last_change":"2024-09-05T02:00:21.416649+0000","last_active":"2024-09-05T02:00:25.423072+0000","last_peered":"2024-09-05T02:00:25.423072+0000","last_clean":"2024-09-05T02:00:25.423072+0000","last_became_active":"2024-09-05T02:00:21.416247+0000","last_became_peered":"2024-09-05T02:00:21.416247+0000","last_unstale":"2024-09-05T02:00:25.423072+0000","last_undegraded":"2024-09-05T02:00:25.423072+0000","last_fullsized":"2024-09-05T02:00:25.423072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:10:00.115655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421769+0000","last_change":"2024-09-05T02:00:14.357719+0000","last_active":"2024-09-05T02:00:25.421769+0000","last_peered":"2024-09-05T02:00:25.421769+0000","last_clean":"2024-09-05T02:00:25.421769+0000","last_became_active":"2024-09-05T02:00:14.357211+0000","last_became_peered":"2024-09-05T02:00:14.357211+0000","last_unstale":"2024-09-05T02:00:25.421769+0000","last_undegraded":"2024-09-05T02:00:25.421769+0000","last_fullsized":"2024-09-05T02:00:25.421769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:27:15.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:55.202216+0000","last_change":"2024-09-05T02:00:12.353893+0000","last_active":"2024-09-05T02:00:55.202216+0000","last_peered":"2024-09-05T02:00:55.202216+0000","last_clean":"2024-09-05T02:00:55.202216+0000","last_became_active":"2024-09-05T02:00:12.353531+0000","last_became_peered":"2024-09-05T02:00:12.353531+0000","last_unstale":"2024-09-05T02:00:55.202216+0000","last_undegraded":"2024-09-05T02:00:55.202216+0000","last_fullsized":"2024-09-05T02:00:55.202216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:58.597446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423504+0000","last_change":"2024-09-05T02:00:21.415203+0000","last_active":"2024-09-05T02:00:25.423504+0000","last_peered":"2024-09-05T02:00:25.423504+0000","last_clean":"2024-09-05T02:00:25.423504+0000","last_became_active":"2024-09-05T02:00:21.414767+0000","last_became_peered":"2024-09-05T02:00:21.414767+0000","last_unstale":"2024-09-05T02:00:25.423504+0000","last_undegraded":"2024-09-05T02:00:25.423504+0000","last_fullsized":"2024-09-05T02:00:25.423504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:01:48.398448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416709+0000","last_change":"2024-09-05T02:00:14.369417+0000","last_active":"2024-09-05T02:00:24.416709+0000","last_peered":"2024-09-05T02:00:24.416709+0000","last_clean":"2024-09-05T02:00:24.416709+0000","last_became_active":"2024-09-05T02:00:14.369093+0000","last_became_peered":"2024-09-05T02:00:14.369093+0000","last_unstale":"2024-09-05T02:00:24.416709+0000","last_undegraded":"2024-09-05T02:00:24.416709+0000","last_fullsized":"2024-09-05T02:00:24.416709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:01:34.239300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:57.216177+0000","last_change":"2024-09-05T02:00:12.347945+0000","last_active":"2024-09-05T02:00:57.216177+0000","last_peered":"2024-09-05T02:00:57.216177+0000","last_clean":"2024-09-05T02:00:57.216177+0000","last_became_active":"2024-09-05T02:00:12.347577+0000","last_became_peered":"2024-09-05T02:00:12.347577+0000","last_unstale":"2024-09-05T02:00:57.216177+0000","last_undegraded":"2024-09-05T02:00:57.216177+0000","last_fullsized":"2024-09-05T02:00:57.216177+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:53:23.947159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423153+0000","last_change":"2024-09-05T02:00:21.419070+0000","last_active":"2024-09-05T02:00:25.423153+0000","last_peered":"2024-09-05T02:00:25.423153+0000","last_clean":"2024-09-05T02:00:25.423153+0000","last_became_active":"2024-09-05T02:00:21.418315+0000","last_became_peered":"2024-09-05T02:00:21.418315+0000","last_unstale":"2024-09-05T02:00:25.423153+0000","last_undegraded":"2024-09-05T02:00:25.423153+0000","last_fullsized":"2024-09-05T02:00:25.423153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:17.974047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422211+0000","last_change":"2024-09-05T02:00:14.362965+0000","last_active":"2024-09-05T02:00:25.422211+0000","last_peered":"2024-09-05T02:00:25.422211+0000","last_clean":"2024-09-05T02:00:25.422211+0000","last_became_active":"2024-09-05T02:00:14.361842+0000","last_became_peered":"2024-09-05T02:00:14.361842+0000","last_unstale":"2024-09-05T02:00:25.422211+0000","last_undegraded":"2024-09-05T02:00:25.422211+0000","last_fullsized":"2024-09-05T02:00:25.422211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:45:10.565392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.227725+0000","last_change":"2024-09-05T02:00:12.349368+0000","last_active":"2024-09-05T02:00:58.227725+0000","last_peered":"2024-09-05T02:00:58.227725+0000","last_clean":"2024-09-05T02:00:58.227725+0000","last_became_active":"2024-09-05T02:00:12.348990+0000","last_became_peered":"2024-09-05T02:00:12.348990+0000","last_unstale":"2024-09-05T02:00:58.227725+0000","last_undegraded":"2024-09-05T02:00:58.227725+0000","last_fullsized":"2024-09-05T02:00:58.227725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:45:04.998316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422304+0000","last_change":"2024-09-05T02:00:21.415467+0000","last_active":"2024-09-05T02:00:25.422304+0000","last_peered":"2024-09-05T02:00:25.422304+0000","last_clean":"2024-09-05T02:00:25.422304+0000","last_became_active":"2024-09-05T02:00:21.415138+0000","last_became_peered":"2024-09-05T02:00:21.415138+0000","last_unstale":"2024-09-05T02:00:25.422304+0000","last_undegraded":"2024-09-05T02:00:25.422304+0000","last_fullsized":"2024-09-05T02:00:25.422304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:51.221383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421232+0000","last_change":"2024-09-05T02:00:14.357961+0000","last_active":"2024-09-05T02:00:25.421232+0000","last_peered":"2024-09-05T02:00:25.421232+0000","last_clean":"2024-09-05T02:00:25.421232+0000","last_became_active":"2024-09-05T02:00:14.357624+0000","last_became_peered":"2024-09-05T02:00:14.357624+0000","last_unstale":"2024-09-05T02:00:25.421232+0000","last_undegraded":"2024-09-05T02:00:25.421232+0000","last_fullsized":"2024-09-05T02:00:25.421232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:01.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:55.202613+0000","last_change":"2024-09-05T02:00:12.783304+0000","last_active":"2024-09-05T02:00:55.202613+0000","last_peered":"2024-09-05T02:00:55.202613+0000","last_clean":"2024-09-05T02:00:55.202613+0000","last_became_active":"2024-09-05T02:00:12.782953+0000","last_became_peered":"2024-09-05T02:00:12.782953+0000","last_unstale":"2024-09-05T02:00:55.202613+0000","last_undegraded":"2024-09-05T02:00:55.202613+0000","last_fullsized":"2024-09-05T02:00:55.202613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:50:59.130705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423575+0000","last_change":"2024-09-05T02:00:21.418357+0000","last_active":"2024-09-05T02:00:25.423575+0000","last_peered":"2024-09-05T02:00:25.423575+0000","last_clean":"2024-09-05T02:00:25.423575+0000","last_became_active":"2024-09-05T02:00:21.417779+0000","last_became_peered":"2024-09-05T02:00:21.417779+0000","last_unstale":"2024-09-05T02:00:25.423575+0000","last_undegraded":"2024-09-05T02:00:25.423575+0000","last_fullsized":"2024-09-05T02:00:25.423575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:37:45.423447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421992+0000","last_change":"2024-09-05T02:00:14.362857+0000","last_active":"2024-09-05T02:00:25.421992+0000","last_peered":"2024-09-05T02:00:25.421992+0000","last_clean":"2024-09-05T02:00:25.421992+0000","last_became_active":"2024-09-05T02:00:14.362556+0000","last_became_peered":"2024-09-05T02:00:14.362556+0000","last_unstale":"2024-09-05T02:00:25.421992+0000","last_undegraded":"2024-09-05T02:00:25.421992+0000","last_fullsized":"2024-09-05T02:00:25.421992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:44:03.413055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.228902+0000","last_change":"2024-09-05T02:00:12.512747+0000","last_active":"2024-09-05T02:00:58.228902+0000","last_peered":"2024-09-05T02:00:58.228902+0000","last_clean":"2024-09-05T02:00:58.228902+0000","last_became_active":"2024-09-05T02:00:12.512471+0000","last_became_peered":"2024-09-05T02:00:12.512471+0000","last_unstale":"2024-09-05T02:00:58.228902+0000","last_undegraded":"2024-09-05T02:00:58.228902+0000","last_fullsized":"2024-09-05T02:00:58.228902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:50.235020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416329+0000","last_change":"2024-09-05T02:00:21.413947+0000","last_active":"2024-09-05T02:00:24.416329+0000","last_peered":"2024-09-05T02:00:24.416329+0000","last_clean":"2024-09-05T02:00:24.416329+0000","last_became_active":"2024-09-05T02:00:21.413488+0000","last_became_peered":"2024-09-05T02:00:21.413488+0000","last_unstale":"2024-09-05T02:00:24.416329+0000","last_undegraded":"2024-09-05T02:00:24.416329+0000","last_fullsized":"2024-09-05T02:00:24.416329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:52:16.201325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.417065+0000","last_change":"2024-09-05T02:00:14.367009+0000","last_active":"2024-09-05T02:00:24.417065+0000","last_peered":"2024-09-05T02:00:24.417065+0000","last_clean":"2024-09-05T02:00:24.417065+0000","last_became_active":"2024-09-05T02:00:14.366698+0000","last_became_peered":"2024-09-05T02:00:14.366698+0000","last_unstale":"2024-09-05T02:00:24.417065+0000","last_undegraded":"2024-09-05T02:00:24.417065+0000","last_fullsized":"2024-09-05T02:00:24.417065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:37:43.811300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:06.380715+0000","last_change":"2024-09-05T02:00:12.346918+0000","last_active":"2024-09-05T02:01:06.380715+0000","last_peered":"2024-09-05T02:01:06.380715+0000","last_clean":"2024-09-05T02:01:06.380715+0000","last_became_active":"2024-09-05T02:00:12.346541+0000","last_became_peered":"2024-09-05T02:00:12.346541+0000","last_unstale":"2024-09-05T02:01:06.380715+0000","last_undegraded":"2024-09-05T02:01:06.380715+0000","last_fullsized":"2024-09-05T02:01:06.380715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:50.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416503+0000","last_change":"2024-09-05T02:00:21.413077+0000","last_active":"2024-09-05T02:00:24.416503+0000","last_peered":"2024-09-05T02:00:24.416503+0000","last_clean":"2024-09-05T02:00:24.416503+0000","last_became_active":"2024-09-05T02:00:21.412698+0000","last_became_peered":"2024-09-05T02:00:21.412698+0000","last_unstale":"2024-09-05T02:00:24.416503+0000","last_undegraded":"2024-09-05T02:00:24.416503+0000","last_fullsized":"2024-09-05T02:00:24.416503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:12:18.463614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415862+0000","last_change":"2024-09-05T02:00:14.363072+0000","last_active":"2024-09-05T02:00:24.415862+0000","last_peered":"2024-09-05T02:00:24.415862+0000","last_clean":"2024-09-05T02:00:24.415862+0000","last_became_active":"2024-09-05T02:00:14.361932+0000","last_became_peered":"2024-09-05T02:00:14.361932+0000","last_unstale":"2024-09-05T02:00:24.415862+0000","last_undegraded":"2024-09-05T02:00:24.415862+0000","last_fullsized":"2024-09-05T02:00:24.415862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:20:13.080963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.229875+0000","last_change":"2024-09-05T02:00:12.783825+0000","last_active":"2024-09-05T02:00:58.229875+0000","last_peered":"2024-09-05T02:00:58.229875+0000","last_clean":"2024-09-05T02:00:58.229875+0000","last_became_active":"2024-09-05T02:00:12.783298+0000","last_became_peered":"2024-09-05T02:00:12.783298+0000","last_unstale":"2024-09-05T02:00:58.229875+0000","last_undegraded":"2024-09-05T02:00:58.229875+0000","last_fullsized":"2024-09-05T02:00:58.229875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:00:10.986318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422188+0000","last_change":"2024-09-05T02:00:21.416474+0000","last_active":"2024-09-05T02:00:25.422188+0000","last_peered":"2024-09-05T02:00:25.422188+0000","last_clean":"2024-09-05T02:00:25.422188+0000","last_became_active":"2024-09-05T02:00:21.416006+0000","last_became_peered":"2024-09-05T02:00:21.416006+0000","last_unstale":"2024-09-05T02:00:25.422188+0000","last_undegraded":"2024-09-05T02:00:25.422188+0000","last_fullsized":"2024-09-05T02:00:25.422188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:00:18.587895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.613698+0000","last_change":"2024-09-05T02:00:14.362458+0000","last_active":"2024-09-05T02:00:24.613698+0000","last_peered":"2024-09-05T02:00:24.613698+0000","last_clean":"2024-09-05T02:00:24.613698+0000","last_became_active":"2024-09-05T02:00:14.362080+0000","last_became_peered":"2024-09-05T02:00:14.362080+0000","last_unstale":"2024-09-05T02:00:24.613698+0000","last_undegraded":"2024-09-05T02:00:24.613698+0000","last_fullsized":"2024-09-05T02:00:24.613698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:47:49.498468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421986+0000","last_change":"2024-09-05T02:00:12.783445+0000","last_active":"2024-09-05T02:00:25.421986+0000","last_peered":"2024-09-05T02:00:25.421986+0000","last_clean":"2024-09-05T02:00:25.421986+0000","last_became_active":"2024-09-05T02:00:12.783131+0000","last_became_peered":"2024-09-05T02:00:12.783131+0000","last_unstale":"2024-09-05T02:00:25.421986+0000","last_undegraded":"2024-09-05T02:00:25.421986+0000","last_fullsized":"2024-09-05T02:00:25.421986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:21:57.743629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421405+0000","last_change":"2024-09-05T02:00:21.418568+0000","last_active":"2024-09-05T02:00:25.421405+0000","last_peered":"2024-09-05T02:00:25.421405+0000","last_clean":"2024-09-05T02:00:25.421405+0000","last_became_active":"2024-09-05T02:00:21.418152+0000","last_became_peered":"2024-09-05T02:00:21.418152+0000","last_unstale":"2024-09-05T02:00:25.421405+0000","last_undegraded":"2024-09-05T02:00:25.421405+0000","last_fullsized":"2024-09-05T02:00:25.421405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:50:17.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.613582+0000","last_change":"2024-09-05T02:00:14.364556+0000","last_active":"2024-09-05T02:00:24.613582+0000","last_peered":"2024-09-05T02:00:24.613582+0000","last_clean":"2024-09-05T02:00:24.613582+0000","last_became_active":"2024-09-05T02:00:14.364285+0000","last_became_peered":"2024-09-05T02:00:14.364285+0000","last_unstale":"2024-09-05T02:00:24.613582+0000","last_undegraded":"2024-09-05T02:00:24.613582+0000","last_fullsized":"2024-09-05T02:00:24.613582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:24:54.901510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:05.251308+0000","last_change":"2024-09-05T02:00:12.347993+0000","last_active":"2024-09-05T02:01:05.251308+0000","last_peered":"2024-09-05T02:01:05.251308+0000","last_clean":"2024-09-05T02:01:05.251308+0000","last_became_active":"2024-09-05T02:00:12.347656+0000","last_became_peered":"2024-09-05T02:00:12.347656+0000","last_unstale":"2024-09-05T02:01:05.251308+0000","last_undegraded":"2024-09-05T02:01:05.251308+0000","last_fullsized":"2024-09-05T02:01:05.251308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:31:23.820379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415609+0000","last_change":"2024-09-05T02:00:21.410786+0000","last_active":"2024-09-05T02:00:24.415609+0000","last_peered":"2024-09-05T02:00:24.415609+0000","last_clean":"2024-09-05T02:00:24.415609+0000","last_became_active":"2024-09-05T02:00:21.410440+0000","last_became_peered":"2024-09-05T02:00:21.410440+0000","last_unstale":"2024-09-05T02:00:24.415609+0000","last_undegraded":"2024-09-05T02:00:24.415609+0000","last_fullsized":"2024-09-05T02:00:24.415609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:22:14.644413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690706+0000","last_change":"2024-09-05T02:00:14.357893+0000","last_active":"2024-09-05T02:00:24.690706+0000","last_peered":"2024-09-05T02:00:24.690706+0000","last_clean":"2024-09-05T02:00:24.690706+0000","last_became_active":"2024-09-05T02:00:14.357551+0000","last_became_peered":"2024-09-05T02:00:14.357551+0000","last_unstale":"2024-09-05T02:00:24.690706+0000","last_undegraded":"2024-09-05T02:00:24.690706+0000","last_fullsized":"2024-09-05T02:00:24.690706+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:04:15.466460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:07.258092+0000","last_change":"2024-09-05T02:00:12.783358+0000","last_active":"2024-09-05T02:01:07.258092+0000","last_peered":"2024-09-05T02:01:07.258092+0000","last_clean":"2024-09-05T02:01:07.258092+0000","last_became_active":"2024-09-05T02:00:12.782950+0000","last_became_peered":"2024-09-05T02:00:12.782950+0000","last_unstale":"2024-09-05T02:01:07.258092+0000","last_undegraded":"2024-09-05T02:01:07.258092+0000","last_fullsized":"2024-09-05T02:01:07.258092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:17:38.823019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415727+0000","last_change":"2024-09-05T02:00:21.416389+0000","last_active":"2024-09-05T02:00:24.415727+0000","last_peered":"2024-09-05T02:00:24.415727+0000","last_clean":"2024-09-05T02:00:24.415727+0000","last_became_active":"2024-09-05T02:00:21.416065+0000","last_became_peered":"2024-09-05T02:00:21.416065+0000","last_unstale":"2024-09-05T02:00:24.415727+0000","last_undegraded":"2024-09-05T02:00:24.415727+0000","last_fullsized":"2024-09-05T02:00:24.415727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:15.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415274+0000","last_change":"2024-09-05T02:00:14.361404+0000","last_active":"2024-09-05T02:00:24.415274+0000","last_peered":"2024-09-05T02:00:24.415274+0000","last_clean":"2024-09-05T02:00:24.415274+0000","last_became_active":"2024-09-05T02:00:14.361031+0000","last_became_peered":"2024-09-05T02:00:14.361031+0000","last_unstale":"2024-09-05T02:00:24.415274+0000","last_undegraded":"2024-09-05T02:00:24.415274+0000","last_fullsized":"2024-09-05T02:00:24.415274+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:40.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:56.206346+0000","last_change":"2024-09-05T02:00:12.351656+0000","last_active":"2024-09-05T02:00:56.206346+0000","last_peered":"2024-09-05T02:00:56.206346+0000","last_clean":"2024-09-05T02:00:56.206346+0000","last_became_active":"2024-09-05T02:00:12.351310+0000","last_became_peered":"2024-09-05T02:00:12.351310+0000","last_unstale":"2024-09-05T02:00:56.206346+0000","last_undegraded":"2024-09-05T02:00:56.206346+0000","last_fullsized":"2024-09-05T02:00:56.206346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:55:27.949471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.691213+0000","last_change":"2024-09-05T02:00:21.417300+0000","last_active":"2024-09-05T02:00:24.691213+0000","last_peered":"2024-09-05T02:00:24.691213+0000","last_clean":"2024-09-05T02:00:24.691213+0000","last_became_active":"2024-09-05T02:00:21.415401+0000","last_became_peered":"2024-09-05T02:00:21.415401+0000","last_unstale":"2024-09-05T02:00:24.691213+0000","last_undegraded":"2024-09-05T02:00:24.691213+0000","last_fullsized":"2024-09-05T02:00:24.691213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:44:02.874857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416841+0000","last_change":"2024-09-05T02:00:14.359356+0000","last_active":"2024-09-05T02:00:24.416841+0000","last_peered":"2024-09-05T02:00:24.416841+0000","last_clean":"2024-09-05T02:00:24.416841+0000","last_became_active":"2024-09-05T02:00:14.359078+0000","last_became_peered":"2024-09-05T02:00:14.359078+0000","last_unstale":"2024-09-05T02:00:24.416841+0000","last_undegraded":"2024-09-05T02:00:24.416841+0000","last_fullsized":"2024-09-05T02:00:24.416841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:40:24.219962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:59.240636+0000","last_change":"2024-09-05T02:00:12.349749+0000","last_active":"2024-09-05T02:00:59.240636+0000","last_peered":"2024-09-05T02:00:59.240636+0000","last_clean":"2024-09-05T02:00:59.240636+0000","last_became_active":"2024-09-05T02:00:12.349247+0000","last_became_peered":"2024-09-05T02:00:12.349247+0000","last_unstale":"2024-09-05T02:00:59.240636+0000","last_undegraded":"2024-09-05T02:00:59.240636+0000","last_fullsized":"2024-09-05T02:00:59.240636+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:04:26.361209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":343,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422612+0000","last_change":"2024-09-05T01:58:57.552989+0000","last_active":"2024-09-05T02:00:25.422612+0000","last_peered":"2024-09-05T02:00:25.422612+0000","last_clean":"2024-09-05T02:00:25.422612+0000","last_became_active":"2024-09-05T01:58:57.552511+0000","last_became_peered":"2024-09-05T01:58:57.552511+0000","last_unstale":"2024-09-05T02:00:25.422612+0000","last_undegraded":"2024-09-05T02:00:25.422612+0000","last_fullsized":"2024-09-05T02:00:25.422612+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_clean_scrub_stamp":"2024-09-05T01:55:35.678445+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:53.711210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422392+0000","last_change":"2024-09-05T02:00:21.421456+0000","last_active":"2024-09-05T02:00:25.422392+0000","last_peered":"2024-09-05T02:00:25.422392+0000","last_clean":"2024-09-05T02:00:25.422392+0000","last_became_active":"2024-09-05T02:00:21.421074+0000","last_became_peered":"2024-09-05T02:00:21.421074+0000","last_unstale":"2024-09-05T02:00:25.422392+0000","last_undegraded":"2024-09-05T02:00:25.422392+0000","last_fullsized":"2024-09-05T02:00:25.422392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:46:00.660079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422568+0000","last_change":"2024-09-05T02:00:14.357638+0000","last_active":"2024-09-05T02:00:25.422568+0000","last_peered":"2024-09-05T02:00:25.422568+0000","last_clean":"2024-09-05T02:00:25.422568+0000","last_became_active":"2024-09-05T02:00:14.357125+0000","last_became_peered":"2024-09-05T02:00:14.357125+0000","last_unstale":"2024-09-05T02:00:25.422568+0000","last_undegraded":"2024-09-05T02:00:25.422568+0000","last_fullsized":"2024-09-05T02:00:25.422568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:02:04.343235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:08.258835+0000","last_change":"2024-09-05T02:00:12.783627+0000","last_active":"2024-09-05T02:01:08.258835+0000","last_peered":"2024-09-05T02:01:08.258835+0000","last_clean":"2024-09-05T02:01:08.258835+0000","last_became_active":"2024-09-05T02:00:12.783112+0000","last_became_peered":"2024-09-05T02:00:12.783112+0000","last_unstale":"2024-09-05T02:01:08.258835+0000","last_undegraded":"2024-09-05T02:01:08.258835+0000","last_fullsized":"2024-09-05T02:01:08.258835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:39:45.190311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421455+0000","last_change":"2024-09-05T02:00:21.413069+0000","last_active":"2024-09-05T02:00:25.421455+0000","last_peered":"2024-09-05T02:00:25.421455+0000","last_clean":"2024-09-05T02:00:25.421455+0000","last_became_active":"2024-09-05T02:00:21.412687+0000","last_became_peered":"2024-09-05T02:00:21.412687+0000","last_unstale":"2024-09-05T02:00:25.421455+0000","last_undegraded":"2024-09-05T02:00:25.421455+0000","last_fullsized":"2024-09-05T02:00:25.421455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:28.883071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421766+0000","last_change":"2024-09-05T02:00:14.363197+0000","last_active":"2024-09-05T02:00:25.421766+0000","last_peered":"2024-09-05T02:00:25.421766+0000","last_clean":"2024-09-05T02:00:25.421766+0000","last_became_active":"2024-09-05T02:00:14.362854+0000","last_became_peered":"2024-09-05T02:00:14.362854+0000","last_unstale":"2024-09-05T02:00:25.421766+0000","last_undegraded":"2024-09-05T02:00:25.421766+0000","last_fullsized":"2024-09-05T02:00:25.421766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:29:28.770604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.231039+0000","last_change":"2024-09-05T02:00:12.348715+0000","last_active":"2024-09-05T02:00:58.231039+0000","last_peered":"2024-09-05T02:00:58.231039+0000","last_clean":"2024-09-05T02:00:58.231039+0000","last_became_active":"2024-09-05T02:00:12.348322+0000","last_became_peered":"2024-09-05T02:00:12.348322+0000","last_unstale":"2024-09-05T02:00:58.231039+0000","last_undegraded":"2024-09-05T02:00:58.231039+0000","last_fullsized":"2024-09-05T02:00:58.231039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:33:55.551080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421146+0000","last_change":"2024-09-05T02:00:21.414103+0000","last_active":"2024-09-05T02:00:25.421146+0000","last_peered":"2024-09-05T02:00:25.421146+0000","last_clean":"2024-09-05T02:00:25.421146+0000","last_became_active":"2024-09-05T02:00:21.413706+0000","last_became_peered":"2024-09-05T02:00:21.413706+0000","last_unstale":"2024-09-05T02:00:25.421146+0000","last_undegraded":"2024-09-05T02:00:25.421146+0000","last_fullsized":"2024-09-05T02:00:25.421146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:46:41.226053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421868+0000","last_change":"2024-09-05T02:00:14.355969+0000","last_active":"2024-09-05T02:00:25.421868+0000","last_peered":"2024-09-05T02:00:25.421868+0000","last_clean":"2024-09-05T02:00:25.421868+0000","last_became_active":"2024-09-05T02:00:14.355581+0000","last_became_peered":"2024-09-05T02:00:14.355581+0000","last_unstale":"2024-09-05T02:00:25.421868+0000","last_undegraded":"2024-09-05T02:00:25.421868+0000","last_fullsized":"2024-09-05T02:00:25.421868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:55:22.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.229946+0000","last_change":"2024-09-05T02:00:12.348148+0000","last_active":"2024-09-05T02:00:58.229946+0000","last_peered":"2024-09-05T02:00:58.229946+0000","last_clean":"2024-09-05T02:00:58.229946+0000","last_became_active":"2024-09-05T02:00:12.347857+0000","last_became_peered":"2024-09-05T02:00:12.347857+0000","last_unstale":"2024-09-05T02:00:58.229946+0000","last_undegraded":"2024-09-05T02:00:58.229946+0000","last_fullsized":"2024-09-05T02:00:58.229946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:22:21.368082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690558+0000","last_change":"2024-09-05T02:00:21.417476+0000","last_active":"2024-09-05T02:00:24.690558+0000","last_peered":"2024-09-05T02:00:24.690558+0000","last_clean":"2024-09-05T02:00:24.690558+0000","last_became_active":"2024-09-05T02:00:21.417036+0000","last_became_peered":"2024-09-05T02:00:21.417036+0000","last_unstale":"2024-09-05T02:00:24.690558+0000","last_undegraded":"2024-09-05T02:00:24.690558+0000","last_fullsized":"2024-09-05T02:00:24.690558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:15:22.510948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422449+0000","last_change":"2024-09-05T02:00:14.358542+0000","last_active":"2024-09-05T02:00:25.422449+0000","last_peered":"2024-09-05T02:00:25.422449+0000","last_clean":"2024-09-05T02:00:25.422449+0000","last_became_active":"2024-09-05T02:00:14.358228+0000","last_became_peered":"2024-09-05T02:00:14.358228+0000","last_unstale":"2024-09-05T02:00:25.422449+0000","last_undegraded":"2024-09-05T02:00:25.422449+0000","last_fullsized":"2024-09-05T02:00:25.422449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:01.627453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.202513+0000","last_change":"2024-09-05T02:00:12.345887+0000","last_active":"2024-09-05T02:00:58.202513+0000","last_peered":"2024-09-05T02:00:58.202513+0000","last_clean":"2024-09-05T02:00:58.202513+0000","last_became_active":"2024-09-05T02:00:12.345491+0000","last_became_peered":"2024-09-05T02:00:12.345491+0000","last_unstale":"2024-09-05T02:00:58.202513+0000","last_undegraded":"2024-09-05T02:00:58.202513+0000","last_fullsized":"2024-09-05T02:00:58.202513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:10:32.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690649+0000","last_change":"2024-09-05T02:00:21.418210+0000","last_active":"2024-09-05T02:00:24.690649+0000","last_peered":"2024-09-05T02:00:24.690649+0000","last_clean":"2024-09-05T02:00:24.690649+0000","last_became_active":"2024-09-05T02:00:21.417769+0000","last_became_peered":"2024-09-05T02:00:21.417769+0000","last_unstale":"2024-09-05T02:00:24.690649+0000","last_undegraded":"2024-09-05T02:00:24.690649+0000","last_fullsized":"2024-09-05T02:00:24.690649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:05.050499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690593+0000","last_change":"2024-09-05T02:00:14.367555+0000","last_active":"2024-09-05T02:00:24.690593+0000","last_peered":"2024-09-05T02:00:24.690593+0000","last_clean":"2024-09-05T02:00:24.690593+0000","last_became_active":"2024-09-05T02:00:14.367222+0000","last_became_peered":"2024-09-05T02:00:14.367222+0000","last_unstale":"2024-09-05T02:00:24.690593+0000","last_undegraded":"2024-09-05T02:00:24.690593+0000","last_fullsized":"2024-09-05T02:00:24.690593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:04:26.172141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.230148+0000","last_change":"2024-09-05T02:00:12.785867+0000","last_active":"2024-09-05T02:00:58.230148+0000","last_peered":"2024-09-05T02:00:58.230148+0000","last_clean":"2024-09-05T02:00:58.230148+0000","last_became_active":"2024-09-05T02:00:12.785470+0000","last_became_peered":"2024-09-05T02:00:12.785470+0000","last_unstale":"2024-09-05T02:00:58.230148+0000","last_undegraded":"2024-09-05T02:00:58.230148+0000","last_fullsized":"2024-09-05T02:00:58.230148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:57:29.367257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415933+0000","last_change":"2024-09-05T02:00:21.412374+0000","last_active":"2024-09-05T02:00:24.415933+0000","last_peered":"2024-09-05T02:00:24.415933+0000","last_clean":"2024-09-05T02:00:24.415933+0000","last_became_active":"2024-09-05T02:00:21.411926+0000","last_became_peered":"2024-09-05T02:00:21.411926+0000","last_unstale":"2024-09-05T02:00:24.415933+0000","last_undegraded":"2024-09-05T02:00:24.415933+0000","last_fullsized":"2024-09-05T02:00:24.415933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:38:29.657795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422370+0000","last_change":"2024-09-05T02:00:14.361332+0000","last_active":"2024-09-05T02:00:25.422370+0000","last_peered":"2024-09-05T02:00:25.422370+0000","last_clean":"2024-09-05T02:00:25.422370+0000","last_became_active":"2024-09-05T02:00:14.360995+0000","last_became_peered":"2024-09-05T02:00:14.360995+0000","last_unstale":"2024-09-05T02:00:25.422370+0000","last_undegraded":"2024-09-05T02:00:25.422370+0000","last_fullsized":"2024-09-05T02:00:25.422370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:12:11.104876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:07.258582+0000","last_change":"2024-09-05T02:00:12.349175+0000","last_active":"2024-09-05T02:01:07.258582+0000","last_peered":"2024-09-05T02:01:07.258582+0000","last_clean":"2024-09-05T02:01:07.258582+0000","last_became_active":"2024-09-05T02:00:12.348647+0000","last_became_peered":"2024-09-05T02:00:12.348647+0000","last_unstale":"2024-09-05T02:01:07.258582+0000","last_undegraded":"2024-09-05T02:01:07.258582+0000","last_fullsized":"2024-09-05T02:01:07.258582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:38:13.363462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422455+0000","last_change":"2024-09-05T02:00:21.414252+0000","last_active":"2024-09-05T02:00:25.422455+0000","last_peered":"2024-09-05T02:00:25.422455+0000","last_clean":"2024-09-05T02:00:25.422455+0000","last_became_active":"2024-09-05T02:00:21.412483+0000","last_became_peered":"2024-09-05T02:00:21.412483+0000","last_unstale":"2024-09-05T02:00:25.422455+0000","last_undegraded":"2024-09-05T02:00:25.422455+0000","last_fullsized":"2024-09-05T02:00:25.422455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:58.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422241+0000","last_change":"2024-09-05T02:00:14.361018+0000","last_active":"2024-09-05T02:00:25.422241+0000","last_peered":"2024-09-05T02:00:25.422241+0000","last_clean":"2024-09-05T02:00:25.422241+0000","last_became_active":"2024-09-05T02:00:14.360591+0000","last_became_peered":"2024-09-05T02:00:14.360591+0000","last_unstale":"2024-09-05T02:00:25.422241+0000","last_undegraded":"2024-09-05T02:00:25.422241+0000","last_fullsized":"2024-09-05T02:00:25.422241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:49.175681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.230423+0000","last_change":"2024-09-05T02:00:12.353257+0000","last_active":"2024-09-05T02:00:58.230423+0000","last_peered":"2024-09-05T02:00:58.230423+0000","last_clean":"2024-09-05T02:00:58.230423+0000","last_became_active":"2024-09-05T02:00:12.352906+0000","last_became_peered":"2024-09-05T02:00:12.352906+0000","last_unstale":"2024-09-05T02:00:58.230423+0000","last_undegraded":"2024-09-05T02:00:58.230423+0000","last_fullsized":"2024-09-05T02:00:58.230423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:53:52.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422494+0000","last_change":"2024-09-05T02:00:21.410346+0000","last_active":"2024-09-05T02:00:25.422494+0000","last_peered":"2024-09-05T02:00:25.422494+0000","last_clean":"2024-09-05T02:00:25.422494+0000","last_became_active":"2024-09-05T02:00:21.409946+0000","last_became_peered":"2024-09-05T02:00:21.409946+0000","last_unstale":"2024-09-05T02:00:25.422494+0000","last_undegraded":"2024-09-05T02:00:25.422494+0000","last_fullsized":"2024-09-05T02:00:25.422494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:56.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415514+0000","last_change":"2024-09-05T02:00:14.360265+0000","last_active":"2024-09-05T02:00:24.415514+0000","last_peered":"2024-09-05T02:00:24.415514+0000","last_clean":"2024-09-05T02:00:24.415514+0000","last_became_active":"2024-09-05T02:00:14.359958+0000","last_became_peered":"2024-09-05T02:00:14.359958+0000","last_unstale":"2024-09-05T02:00:24.415514+0000","last_undegraded":"2024-09-05T02:00:24.415514+0000","last_fullsized":"2024-09-05T02:00:24.415514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:15:45.211207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:59.240546+0000","last_change":"2024-09-05T02:00:12.349443+0000","last_active":"2024-09-05T02:00:59.240546+0000","last_peered":"2024-09-05T02:00:59.240546+0000","last_clean":"2024-09-05T02:00:59.240546+0000","last_became_active":"2024-09-05T02:00:12.348988+0000","last_became_peered":"2024-09-05T02:00:12.348988+0000","last_unstale":"2024-09-05T02:00:59.240546+0000","last_undegraded":"2024-09-05T02:00:59.240546+0000","last_fullsized":"2024-09-05T02:00:59.240546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:14:21.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.691110+0000","last_change":"2024-09-05T02:00:21.417662+0000","last_active":"2024-09-05T02:00:24.691110+0000","last_peered":"2024-09-05T02:00:24.691110+0000","last_clean":"2024-09-05T02:00:24.691110+0000","last_became_active":"2024-09-05T02:00:21.417238+0000","last_became_peered":"2024-09-05T02:00:21.417238+0000","last_unstale":"2024-09-05T02:00:24.691110+0000","last_undegraded":"2024-09-05T02:00:24.691110+0000","last_fullsized":"2024-09-05T02:00:24.691110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:05:59.304480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421675+0000","last_change":"2024-09-05T02:00:14.355928+0000","last_active":"2024-09-05T02:00:25.421675+0000","last_peered":"2024-09-05T02:00:25.421675+0000","last_clean":"2024-09-05T02:00:25.421675+0000","last_became_active":"2024-09-05T02:00:14.355576+0000","last_became_peered":"2024-09-05T02:00:14.355576+0000","last_unstale":"2024-09-05T02:00:25.421675+0000","last_undegraded":"2024-09-05T02:00:25.421675+0000","last_fullsized":"2024-09-05T02:00:25.421675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:22:50.095434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'31","reported_seq":87,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:08.689261+0000","last_change":"2024-09-05T02:00:12.352198+0000","last_active":"2024-09-05T02:01:08.689261+0000","last_peered":"2024-09-05T02:01:08.689261+0000","last_clean":"2024-09-05T02:01:08.689261+0000","last_became_active":"2024-09-05T02:00:12.351806+0000","last_became_peered":"2024-09-05T02:00:12.351806+0000","last_unstale":"2024-09-05T02:01:08.689261+0000","last_undegraded":"2024-09-05T02:01:08.689261+0000","last_fullsized":"2024-09-05T02:01:08.689261+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.178160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52408,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":55,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421650+0000","last_change":"2024-09-05T02:00:21.416223+0000","last_active":"2024-09-05T02:00:25.421650+0000","last_peered":"2024-09-05T02:00:25.421650+0000","last_clean":"2024-09-05T02:00:25.421650+0000","last_became_active":"2024-09-05T02:00:21.415610+0000","last_became_peered":"2024-09-05T02:00:21.415610+0000","last_unstale":"2024-09-05T02:00:25.421650+0000","last_undegraded":"2024-09-05T02:00:25.421650+0000","last_fullsized":"2024-09-05T02:00:25.421650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:08:40.955438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421591+0000","last_change":"2024-09-05T02:00:14.365372+0000","last_active":"2024-09-05T02:00:25.421591+0000","last_peered":"2024-09-05T02:00:25.421591+0000","last_clean":"2024-09-05T02:00:25.421591+0000","last_became_active":"2024-09-05T02:00:14.365064+0000","last_became_peered":"2024-09-05T02:00:14.365064+0000","last_unstale":"2024-09-05T02:00:25.421591+0000","last_undegraded":"2024-09-05T02:00:25.421591+0000","last_fullsized":"2024-09-05T02:00:25.421591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:02:29.378126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:57.217191+0000","last_change":"2024-09-05T02:00:12.783487+0000","last_active":"2024-09-05T02:00:57.217191+0000","last_peered":"2024-09-05T02:00:57.217191+0000","last_clean":"2024-09-05T02:00:57.217191+0000","last_became_active":"2024-09-05T02:00:12.783087+0000","last_became_peered":"2024-09-05T02:00:12.783087+0000","last_unstale":"2024-09-05T02:00:57.217191+0000","last_undegraded":"2024-09-05T02:00:57.217191+0000","last_fullsized":"2024-09-05T02:00:57.217191+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:08.574845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423345+0000","last_change":"2024-09-05T02:00:21.419250+0000","last_active":"2024-09-05T02:00:25.423345+0000","last_peered":"2024-09-05T02:00:25.423345+0000","last_clean":"2024-09-05T02:00:25.423345+0000","last_became_active":"2024-09-05T02:00:21.418851+0000","last_became_peered":"2024-09-05T02:00:21.418851+0000","last_unstale":"2024-09-05T02:00:25.423345+0000","last_undegraded":"2024-09-05T02:00:25.423345+0000","last_fullsized":"2024-09-05T02:00:25.423345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:24:46.762006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422278+0000","last_change":"2024-09-05T02:00:14.359172+0000","last_active":"2024-09-05T02:00:25.422278+0000","last_peered":"2024-09-05T02:00:25.422278+0000","last_clean":"2024-09-05T02:00:25.422278+0000","last_became_active":"2024-09-05T02:00:14.358782+0000","last_became_peered":"2024-09-05T02:00:14.358782+0000","last_unstale":"2024-09-05T02:00:25.422278+0000","last_undegraded":"2024-09-05T02:00:25.422278+0000","last_fullsized":"2024-09-05T02:00:25.422278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:06:12.150349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.230240+0000","last_change":"2024-09-05T02:00:12.349362+0000","last_active":"2024-09-05T02:00:58.230240+0000","last_peered":"2024-09-05T02:00:58.230240+0000","last_clean":"2024-09-05T02:00:58.230240+0000","last_became_active":"2024-09-05T02:00:12.349016+0000","last_became_peered":"2024-09-05T02:00:12.349016+0000","last_unstale":"2024-09-05T02:00:58.230240+0000","last_undegraded":"2024-09-05T02:00:58.230240+0000","last_fullsized":"2024-09-05T02:00:58.230240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:37.603717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421237+0000","last_change":"2024-09-05T02:00:21.412942+0000","last_active":"2024-09-05T02:00:25.421237+0000","last_peered":"2024-09-05T02:00:25.421237+0000","last_clean":"2024-09-05T02:00:25.421237+0000","last_became_active":"2024-09-05T02:00:21.412057+0000","last_became_peered":"2024-09-05T02:00:21.412057+0000","last_unstale":"2024-09-05T02:00:25.421237+0000","last_undegraded":"2024-09-05T02:00:25.421237+0000","last_fullsized":"2024-09-05T02:00:25.421237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:59.872818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422277+0000","last_change":"2024-09-05T02:00:14.356636+0000","last_active":"2024-09-05T02:00:25.422277+0000","last_peered":"2024-09-05T02:00:25.422277+0000","last_clean":"2024-09-05T02:00:25.422277+0000","last_became_active":"2024-09-05T02:00:14.356321+0000","last_became_peered":"2024-09-05T02:00:14.356321+0000","last_unstale":"2024-09-05T02:00:25.422277+0000","last_undegraded":"2024-09-05T02:00:25.422277+0000","last_fullsized":"2024-09-05T02:00:25.422277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:02.112646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:08.254985+0000","last_change":"2024-09-05T02:00:12.353078+0000","last_active":"2024-09-05T02:01:08.254985+0000","last_peered":"2024-09-05T02:01:08.254985+0000","last_clean":"2024-09-05T02:01:08.254985+0000","last_became_active":"2024-09-05T02:00:12.352707+0000","last_became_peered":"2024-09-05T02:00:12.352707+0000","last_unstale":"2024-09-05T02:01:08.254985+0000","last_undegraded":"2024-09-05T02:01:08.254985+0000","last_fullsized":"2024-09-05T02:01:08.254985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:28:29.934304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416652+0000","last_change":"2024-09-05T02:00:21.421362+0000","last_active":"2024-09-05T02:00:24.416652+0000","last_peered":"2024-09-05T02:00:24.416652+0000","last_clean":"2024-09-05T02:00:24.416652+0000","last_became_active":"2024-09-05T02:00:21.420998+0000","last_became_peered":"2024-09-05T02:00:21.420998+0000","last_unstale":"2024-09-05T02:00:24.416652+0000","last_undegraded":"2024-09-05T02:00:24.416652+0000","last_fullsized":"2024-09-05T02:00:24.416652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:26:14.105489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421794+0000","last_change":"2024-09-05T02:00:21.416703+0000","last_active":"2024-09-05T02:00:25.421794+0000","last_peered":"2024-09-05T02:00:25.421794+0000","last_clean":"2024-09-05T02:00:25.421794+0000","last_became_active":"2024-09-05T02:00:21.415913+0000","last_became_peered":"2024-09-05T02:00:21.415913+0000","last_unstale":"2024-09-05T02:00:25.421794+0000","last_undegraded":"2024-09-05T02:00:25.421794+0000","last_fullsized":"2024-09-05T02:00:25.421794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:56.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421356+0000","last_change":"2024-09-05T02:00:14.355658+0000","last_active":"2024-09-05T02:00:25.421356+0000","last_peered":"2024-09-05T02:00:25.421356+0000","last_clean":"2024-09-05T02:00:25.421356+0000","last_became_active":"2024-09-05T02:00:14.355087+0000","last_became_peered":"2024-09-05T02:00:14.355087+0000","last_unstale":"2024-09-05T02:00:25.421356+0000","last_undegraded":"2024-09-05T02:00:25.421356+0000","last_fullsized":"2024-09-05T02:00:25.421356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:58:08.369027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.229348+0000","last_change":"2024-09-05T02:00:12.348171+0000","last_active":"2024-09-05T02:00:58.229348+0000","last_peered":"2024-09-05T02:00:58.229348+0000","last_clean":"2024-09-05T02:00:58.229348+0000","last_became_active":"2024-09-05T02:00:12.347760+0000","last_became_peered":"2024-09-05T02:00:12.347760+0000","last_unstale":"2024-09-05T02:00:58.229348+0000","last_undegraded":"2024-09-05T02:00:58.229348+0000","last_fullsized":"2024-09-05T02:00:58.229348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:03:43.748458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422739+0000","last_change":"2024-09-05T02:00:21.413015+0000","last_active":"2024-09-05T02:00:25.422739+0000","last_peered":"2024-09-05T02:00:25.422739+0000","last_clean":"2024-09-05T02:00:25.422739+0000","last_became_active":"2024-09-05T02:00:21.412631+0000","last_became_peered":"2024-09-05T02:00:21.412631+0000","last_unstale":"2024-09-05T02:00:25.422739+0000","last_undegraded":"2024-09-05T02:00:25.422739+0000","last_fullsized":"2024-09-05T02:00:25.422739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:35.094273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416602+0000","last_change":"2024-09-05T02:00:14.354552+0000","last_active":"2024-09-05T02:00:24.416602+0000","last_peered":"2024-09-05T02:00:24.416602+0000","last_clean":"2024-09-05T02:00:24.416602+0000","last_became_active":"2024-09-05T02:00:14.354150+0000","last_became_peered":"2024-09-05T02:00:14.354150+0000","last_unstale":"2024-09-05T02:00:24.416602+0000","last_undegraded":"2024-09-05T02:00:24.416602+0000","last_fullsized":"2024-09-05T02:00:24.416602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:16:19.836148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:56.206744+0000","last_change":"2024-09-05T02:00:12.342608+0000","last_active":"2024-09-05T02:00:56.206744+0000","last_peered":"2024-09-05T02:00:56.206744+0000","last_clean":"2024-09-05T02:00:56.206744+0000","last_became_active":"2024-09-05T02:00:12.342235+0000","last_became_peered":"2024-09-05T02:00:12.342235+0000","last_unstale":"2024-09-05T02:00:56.206744+0000","last_undegraded":"2024-09-05T02:00:56.206744+0000","last_fullsized":"2024-09-05T02:00:56.206744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:54.568963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:57.217311+0000","last_change":"2024-09-05T02:00:12.352247+0000","last_active":"2024-09-05T02:00:57.217311+0000","last_peered":"2024-09-05T02:00:57.217311+0000","last_clean":"2024-09-05T02:00:57.217311+0000","last_became_active":"2024-09-05T02:00:12.351897+0000","last_became_peered":"2024-09-05T02:00:12.351897+0000","last_unstale":"2024-09-05T02:00:57.217311+0000","last_undegraded":"2024-09-05T02:00:57.217311+0000","last_fullsized":"2024-09-05T02:00:57.217311+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:48.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690897+0000","last_change":"2024-09-05T02:00:14.366415+0000","last_active":"2024-09-05T02:00:24.690897+0000","last_peered":"2024-09-05T02:00:24.690897+0000","last_clean":"2024-09-05T02:00:24.690897+0000","last_became_active":"2024-09-05T02:00:14.366103+0000","last_became_peered":"2024-09-05T02:00:14.366103+0000","last_unstale":"2024-09-05T02:00:24.690897+0000","last_undegraded":"2024-09-05T02:00:24.690897+0000","last_fullsized":"2024-09-05T02:00:24.690897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:22:21.585867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416506+0000","last_change":"2024-09-05T02:00:21.416879+0000","last_active":"2024-09-05T02:00:24.416506+0000","last_peered":"2024-09-05T02:00:24.416506+0000","last_clean":"2024-09-05T02:00:24.416506+0000","last_became_active":"2024-09-05T02:00:21.415887+0000","last_became_peered":"2024-09-05T02:00:21.415887+0000","last_unstale":"2024-09-05T02:00:24.416506+0000","last_undegraded":"2024-09-05T02:00:24.416506+0000","last_fullsized":"2024-09-05T02:00:24.416506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:58.106529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:08.255511+0000","last_change":"2024-09-05T02:00:12.354055+0000","last_active":"2024-09-05T02:01:08.255511+0000","last_peered":"2024-09-05T02:01:08.255511+0000","last_clean":"2024-09-05T02:01:08.255511+0000","last_became_active":"2024-09-05T02:00:12.353754+0000","last_became_peered":"2024-09-05T02:00:12.353754+0000","last_unstale":"2024-09-05T02:01:08.255511+0000","last_undegraded":"2024-09-05T02:01:08.255511+0000","last_fullsized":"2024-09-05T02:01:08.255511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:17.671079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421856+0000","last_change":"2024-09-05T02:00:14.358646+0000","last_active":"2024-09-05T02:00:25.421856+0000","last_peered":"2024-09-05T02:00:25.421856+0000","last_clean":"2024-09-05T02:00:25.421856+0000","last_became_active":"2024-09-05T02:00:14.358337+0000","last_became_peered":"2024-09-05T02:00:14.358337+0000","last_unstale":"2024-09-05T02:00:25.421856+0000","last_undegraded":"2024-09-05T02:00:25.421856+0000","last_fullsized":"2024-09-05T02:00:25.421856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:20:39.976731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416799+0000","last_change":"2024-09-05T02:00:21.416611+0000","last_active":"2024-09-05T02:00:24.416799+0000","last_peered":"2024-09-05T02:00:24.416799+0000","last_clean":"2024-09-05T02:00:24.416799+0000","last_became_active":"2024-09-05T02:00:21.416152+0000","last_became_peered":"2024-09-05T02:00:21.416152+0000","last_unstale":"2024-09-05T02:00:24.416799+0000","last_undegraded":"2024-09-05T02:00:24.416799+0000","last_fullsized":"2024-09-05T02:00:24.416799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:20.670383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421422+0000","last_change":"2024-09-05T02:00:14.359015+0000","last_active":"2024-09-05T02:00:25.421422+0000","last_peered":"2024-09-05T02:00:25.421422+0000","last_clean":"2024-09-05T02:00:25.421422+0000","last_became_active":"2024-09-05T02:00:14.358642+0000","last_became_peered":"2024-09-05T02:00:14.358642+0000","last_unstale":"2024-09-05T02:00:25.421422+0000","last_undegraded":"2024-09-05T02:00:25.421422+0000","last_fullsized":"2024-09-05T02:00:25.421422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:18:03.723033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416457+0000","last_change":"2024-09-05T02:00:14.362391+0000","last_active":"2024-09-05T02:00:24.416457+0000","last_peered":"2024-09-05T02:00:24.416457+0000","last_clean":"2024-09-05T02:00:24.416457+0000","last_became_active":"2024-09-05T02:00:14.362007+0000","last_became_peered":"2024-09-05T02:00:14.362007+0000","last_unstale":"2024-09-05T02:00:24.416457+0000","last_undegraded":"2024-09-05T02:00:24.416457+0000","last_fullsized":"2024-09-05T02:00:24.416457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:39:02.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416177+0000","last_change":"2024-09-05T02:00:14.362769+0000","last_active":"2024-09-05T02:00:24.416177+0000","last_peered":"2024-09-05T02:00:24.416177+0000","last_clean":"2024-09-05T02:00:24.416177+0000","last_became_active":"2024-09-05T02:00:14.362310+0000","last_became_peered":"2024-09-05T02:00:14.362310+0000","last_unstale":"2024-09-05T02:00:24.416177+0000","last_undegraded":"2024-09-05T02:00:24.416177+0000","last_fullsized":"2024-09-05T02:00:24.416177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:27:11.288158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421122+0000","last_change":"2024-09-05T02:00:14.357820+0000","last_active":"2024-09-05T02:00:25.421122+0000","last_peered":"2024-09-05T02:00:25.421122+0000","last_clean":"2024-09-05T02:00:25.421122+0000","last_became_active":"2024-09-05T02:00:14.357320+0000","last_became_peered":"2024-09-05T02:00:14.357320+0000","last_unstale":"2024-09-05T02:00:25.421122+0000","last_undegraded":"2024-09-05T02:00:25.421122+0000","last_fullsized":"2024-09-05T02:00:25.421122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:39.105192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415760+0000","last_change":"2024-09-05T02:00:14.356726+0000","last_active":"2024-09-05T02:00:24.415760+0000","last_peered":"2024-09-05T02:00:24.415760+0000","last_clean":"2024-09-05T02:00:24.415760+0000","last_became_active":"2024-09-05T02:00:14.356452+0000","last_became_peered":"2024-09-05T02:00:14.356452+0000","last_unstale":"2024-09-05T02:00:24.415760+0000","last_undegraded":"2024-09-05T02:00:24.415760+0000","last_fullsized":"2024-09-05T02:00:24.415760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:34:36.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421967+0000","last_change":"2024-09-05T02:00:14.357645+0000","last_active":"2024-09-05T02:00:25.421967+0000","last_peered":"2024-09-05T02:00:25.421967+0000","last_clean":"2024-09-05T02:00:25.421967+0000","last_became_active":"2024-09-05T02:00:14.357330+0000","last_became_peered":"2024-09-05T02:00:14.357330+0000","last_unstale":"2024-09-05T02:00:25.421967+0000","last_undegraded":"2024-09-05T02:00:25.421967+0000","last_fullsized":"2024-09-05T02:00:25.421967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:42.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416761+0000","last_change":"2024-09-05T02:00:14.363413+0000","last_active":"2024-09-05T02:00:24.416761+0000","last_peered":"2024-09-05T02:00:24.416761+0000","last_clean":"2024-09-05T02:00:24.416761+0000","last_became_active":"2024-09-05T02:00:14.362821+0000","last_became_peered":"2024-09-05T02:00:14.362821+0000","last_unstale":"2024-09-05T02:00:24.416761+0000","last_undegraded":"2024-09-05T02:00:24.416761+0000","last_fullsized":"2024-09-05T02:00:24.416761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:49:39.084171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.417022+0000","last_change":"2024-09-05T02:00:14.366581+0000","last_active":"2024-09-05T02:00:24.417022+0000","last_peered":"2024-09-05T02:00:24.417022+0000","last_clean":"2024-09-05T02:00:24.417022+0000","last_became_active":"2024-09-05T02:00:14.366287+0000","last_became_peered":"2024-09-05T02:00:14.366287+0000","last_unstale":"2024-09-05T02:00:24.417022+0000","last_undegraded":"2024-09-05T02:00:24.417022+0000","last_fullsized":"2024-09-05T02:00:24.417022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:36:57.143489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690488+0000","last_change":"2024-09-05T02:00:14.360311+0000","last_active":"2024-09-05T02:00:24.690488+0000","last_peered":"2024-09-05T02:00:24.690488+0000","last_clean":"2024-09-05T02:00:24.690488+0000","last_became_active":"2024-09-05T02:00:14.360028+0000","last_became_peered":"2024-09-05T02:00:14.360028+0000","last_unstale":"2024-09-05T02:00:24.690488+0000","last_undegraded":"2024-09-05T02:00:24.690488+0000","last_fullsized":"2024-09-05T02:00:24.690488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:35:38.804555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416656+0000","last_change":"2024-09-05T02:00:14.365853+0000","last_active":"2024-09-05T02:00:24.416656+0000","last_peered":"2024-09-05T02:00:24.416656+0000","last_clean":"2024-09-05T02:00:24.416656+0000","last_became_active":"2024-09-05T02:00:14.365549+0000","last_became_peered":"2024-09-05T02:00:14.365549+0000","last_unstale":"2024-09-05T02:00:24.416656+0000","last_undegraded":"2024-09-05T02:00:24.416656+0000","last_fullsized":"2024-09-05T02:00:24.416656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:58:45.897592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421705+0000","last_change":"2024-09-05T02:00:14.367357+0000","last_active":"2024-09-05T02:00:25.421705+0000","last_peered":"2024-09-05T02:00:25.421705+0000","last_clean":"2024-09-05T02:00:25.421705+0000","last_became_active":"2024-09-05T02:00:14.366794+0000","last_became_peered":"2024-09-05T02:00:14.366794+0000","last_unstale":"2024-09-05T02:00:25.421705+0000","last_undegraded":"2024-09-05T02:00:25.421705+0000","last_fullsized":"2024-09-05T02:00:25.421705+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:40:51.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415730+0000","last_change":"2024-09-05T02:00:14.356991+0000","last_active":"2024-09-05T02:00:24.415730+0000","last_peered":"2024-09-05T02:00:24.415730+0000","last_clean":"2024-09-05T02:00:24.415730+0000","last_became_active":"2024-09-05T02:00:14.356662+0000","last_became_peered":"2024-09-05T02:00:14.356662+0000","last_unstale":"2024-09-05T02:00:24.415730+0000","last_undegraded":"2024-09-05T02:00:24.415730+0000","last_fullsized":"2024-09-05T02:00:24.415730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:39:53.087448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422572+0000","last_change":"2024-09-05T02:00:14.357750+0000","last_active":"2024-09-05T02:00:25.422572+0000","last_peered":"2024-09-05T02:00:25.422572+0000","last_clean":"2024-09-05T02:00:25.422572+0000","last_became_active":"2024-09-05T02:00:14.357413+0000","last_became_peered":"2024-09-05T02:00:14.357413+0000","last_unstale":"2024-09-05T02:00:25.422572+0000","last_undegraded":"2024-09-05T02:00:25.422572+0000","last_fullsized":"2024-09-05T02:00:25.422572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:29:47.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416894+0000","last_change":"2024-09-05T02:00:14.364068+0000","last_active":"2024-09-05T02:00:24.416894+0000","last_peered":"2024-09-05T02:00:24.416894+0000","last_clean":"2024-09-05T02:00:24.416894+0000","last_became_active":"2024-09-05T02:00:14.363674+0000","last_became_peered":"2024-09-05T02:00:14.363674+0000","last_unstale":"2024-09-05T02:00:24.416894+0000","last_undegraded":"2024-09-05T02:00:24.416894+0000","last_fullsized":"2024-09-05T02:00:24.416894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:28:00.788668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422245+0000","last_change":"2024-09-05T02:00:14.355152+0000","last_active":"2024-09-05T02:00:25.422245+0000","last_peered":"2024-09-05T02:00:25.422245+0000","last_clean":"2024-09-05T02:00:25.422245+0000","last_became_active":"2024-09-05T02:00:14.354874+0000","last_became_peered":"2024-09-05T02:00:14.354874+0000","last_unstale":"2024-09-05T02:00:25.422245+0000","last_undegraded":"2024-09-05T02:00:25.422245+0000","last_fullsized":"2024-09-05T02:00:25.422245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:14:17.069049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422966+0000","last_change":"2024-09-05T02:00:14.360241+0000","last_active":"2024-09-05T02:00:25.422966+0000","last_peered":"2024-09-05T02:00:25.422966+0000","last_clean":"2024-09-05T02:00:25.422966+0000","last_became_active":"2024-09-05T02:00:14.359648+0000","last_became_peered":"2024-09-05T02:00:14.359648+0000","last_unstale":"2024-09-05T02:00:25.422966+0000","last_undegraded":"2024-09-05T02:00:25.422966+0000","last_fullsized":"2024-09-05T02:00:25.422966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:15.060575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421357+0000","last_change":"2024-09-05T02:00:14.366094+0000","last_active":"2024-09-05T02:00:25.421357+0000","last_peered":"2024-09-05T02:00:25.421357+0000","last_clean":"2024-09-05T02:00:25.421357+0000","last_became_active":"2024-09-05T02:00:14.365788+0000","last_became_peered":"2024-09-05T02:00:14.365788+0000","last_unstale":"2024-09-05T02:00:25.421357+0000","last_undegraded":"2024-09-05T02:00:25.421357+0000","last_fullsized":"2024-09-05T02:00:25.421357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:23:00.909487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.420957+0000","last_change":"2024-09-05T02:00:14.364651+0000","last_active":"2024-09-05T02:00:25.420957+0000","last_peered":"2024-09-05T02:00:25.420957+0000","last_clean":"2024-09-05T02:00:25.420957+0000","last_became_active":"2024-09-05T02:00:14.364365+0000","last_became_peered":"2024-09-05T02:00:14.364365+0000","last_unstale":"2024-09-05T02:00:25.420957+0000","last_undegraded":"2024-09-05T02:00:25.420957+0000","last_fullsized":"2024-09-05T02:00:25.420957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:14.978382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.420874+0000","last_change":"2024-09-05T02:00:14.364761+0000","last_active":"2024-09-05T02:00:25.420874+0000","last_peered":"2024-09-05T02:00:25.420874+0000","last_clean":"2024-09-05T02:00:25.420874+0000","last_became_active":"2024-09-05T02:00:14.364204+0000","last_became_peered":"2024-09-05T02:00:14.364204+0000","last_unstale":"2024-09-05T02:00:25.420874+0000","last_undegraded":"2024-09-05T02:00:25.420874+0000","last_fullsized":"2024-09-05T02:00:25.420874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:16:51.076304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415636+0000","last_change":"2024-09-05T02:00:14.357632+0000","last_active":"2024-09-05T02:00:24.415636+0000","last_peered":"2024-09-05T02:00:24.415636+0000","last_clean":"2024-09-05T02:00:24.415636+0000","last_became_active":"2024-09-05T02:00:14.357348+0000","last_became_peered":"2024-09-05T02:00:14.357348+0000","last_unstale":"2024-09-05T02:00:24.415636+0000","last_undegraded":"2024-09-05T02:00:24.415636+0000","last_fullsized":"2024-09-05T02:00:24.415636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:11.163292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423340+0000","last_change":"2024-09-05T02:00:14.361484+0000","last_active":"2024-09-05T02:00:25.423340+0000","last_peered":"2024-09-05T02:00:25.423340+0000","last_clean":"2024-09-05T02:00:25.423340+0000","last_became_active":"2024-09-05T02:00:14.361156+0000","last_became_peered":"2024-09-05T02:00:14.361156+0000","last_unstale":"2024-09-05T02:00:25.423340+0000","last_undegraded":"2024-09-05T02:00:25.423340+0000","last_fullsized":"2024-09-05T02:00:25.423340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:40.124733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422605+0000","last_change":"2024-09-05T02:00:14.358325+0000","last_active":"2024-09-05T02:00:25.422605+0000","last_peered":"2024-09-05T02:00:25.422605+0000","last_clean":"2024-09-05T02:00:25.422605+0000","last_became_active":"2024-09-05T02:00:14.357993+0000","last_became_peered":"2024-09-05T02:00:14.357993+0000","last_unstale":"2024-09-05T02:00:25.422605+0000","last_undegraded":"2024-09-05T02:00:25.422605+0000","last_fullsized":"2024-09-05T02:00:25.422605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:18.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.614040+0000","last_change":"2024-09-05T02:00:14.365291+0000","last_active":"2024-09-05T02:00:24.614040+0000","last_peered":"2024-09-05T02:00:24.614040+0000","last_clean":"2024-09-05T02:00:24.614040+0000","last_became_active":"2024-09-05T02:00:14.364937+0000","last_became_peered":"2024-09-05T02:00:14.364937+0000","last_unstale":"2024-09-05T02:00:24.614040+0000","last_undegraded":"2024-09-05T02:00:24.614040+0000","last_fullsized":"2024-09-05T02:00:24.614040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:42:57.712651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422643+0000","last_change":"2024-09-05T02:00:14.367603+0000","last_active":"2024-09-05T02:00:25.422643+0000","last_peered":"2024-09-05T02:00:25.422643+0000","last_clean":"2024-09-05T02:00:25.422643+0000","last_became_active":"2024-09-05T02:00:14.367267+0000","last_became_peered":"2024-09-05T02:00:14.367267+0000","last_unstale":"2024-09-05T02:00:25.422643+0000","last_undegraded":"2024-09-05T02:00:25.422643+0000","last_fullsized":"2024-09-05T02:00:25.422643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:51:34.595898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416590+0000","last_change":"2024-09-05T02:00:14.369499+0000","last_active":"2024-09-05T02:00:24.416590+0000","last_peered":"2024-09-05T02:00:24.416590+0000","last_clean":"2024-09-05T02:00:24.416590+0000","last_became_active":"2024-09-05T02:00:14.369169+0000","last_became_peered":"2024-09-05T02:00:14.369169+0000","last_unstale":"2024-09-05T02:00:24.416590+0000","last_undegraded":"2024-09-05T02:00:24.416590+0000","last_fullsized":"2024-09-05T02:00:24.416590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:17.161204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422019+0000","last_change":"2024-09-05T02:00:14.360320+0000","last_active":"2024-09-05T02:00:25.422019+0000","last_peered":"2024-09-05T02:00:25.422019+0000","last_clean":"2024-09-05T02:00:25.422019+0000","last_became_active":"2024-09-05T02:00:14.359991+0000","last_became_peered":"2024-09-05T02:00:14.359991+0000","last_unstale":"2024-09-05T02:00:25.422019+0000","last_undegraded":"2024-09-05T02:00:25.422019+0000","last_fullsized":"2024-09-05T02:00:25.422019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:03.721863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.613563+0000","last_change":"2024-09-05T02:00:14.364660+0000","last_active":"2024-09-05T02:00:24.613563+0000","last_peered":"2024-09-05T02:00:24.613563+0000","last_clean":"2024-09-05T02:00:24.613563+0000","last_became_active":"2024-09-05T02:00:14.364313+0000","last_became_peered":"2024-09-05T02:00:14.364313+0000","last_unstale":"2024-09-05T02:00:24.613563+0000","last_undegraded":"2024-09-05T02:00:24.613563+0000","last_fullsized":"2024-09-05T02:00:24.613563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:47.496063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421354+0000","last_change":"2024-09-05T02:00:14.359853+0000","last_active":"2024-09-05T02:00:25.421354+0000","last_peered":"2024-09-05T02:00:25.421354+0000","last_clean":"2024-09-05T02:00:25.421354+0000","last_became_active":"2024-09-05T02:00:14.359543+0000","last_became_peered":"2024-09-05T02:00:14.359543+0000","last_unstale":"2024-09-05T02:00:25.421354+0000","last_undegraded":"2024-09-05T02:00:25.421354+0000","last_fullsized":"2024-09-05T02:00:25.421354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:30:29.415492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423281+0000","last_change":"2024-09-05T02:00:14.363836+0000","last_active":"2024-09-05T02:00:25.423281+0000","last_peered":"2024-09-05T02:00:25.423281+0000","last_clean":"2024-09-05T02:00:25.423281+0000","last_became_active":"2024-09-05T02:00:14.363481+0000","last_became_peered":"2024-09-05T02:00:14.363481+0000","last_unstale":"2024-09-05T02:00:25.423281+0000","last_undegraded":"2024-09-05T02:00:25.423281+0000","last_fullsized":"2024-09-05T02:00:25.423281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:46.443519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421556+0000","last_change":"2024-09-05T02:00:14.367191+0000","last_active":"2024-09-05T02:00:25.421556+0000","last_peered":"2024-09-05T02:00:25.421556+0000","last_clean":"2024-09-05T02:00:25.421556+0000","last_became_active":"2024-09-05T02:00:14.366831+0000","last_became_peered":"2024-09-05T02:00:14.366831+0000","last_unstale":"2024-09-05T02:00:25.421556+0000","last_undegraded":"2024-09-05T02:00:25.421556+0000","last_fullsized":"2024-09-05T02:00:25.421556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:55.690642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421894+0000","last_change":"2024-09-05T02:00:14.355774+0000","last_active":"2024-09-05T02:00:25.421894+0000","last_peered":"2024-09-05T02:00:25.421894+0000","last_clean":"2024-09-05T02:00:25.421894+0000","last_became_active":"2024-09-05T02:00:14.355293+0000","last_became_peered":"2024-09-05T02:00:14.355293+0000","last_unstale":"2024-09-05T02:00:25.421894+0000","last_undegraded":"2024-09-05T02:00:25.421894+0000","last_fullsized":"2024-09-05T02:00:25.421894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:07:00.547820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422874+0000","last_change":"2024-09-05T02:00:14.359978+0000","last_active":"2024-09-05T02:00:25.422874+0000","last_peered":"2024-09-05T02:00:25.422874+0000","last_clean":"2024-09-05T02:00:25.422874+0000","last_became_active":"2024-09-05T02:00:14.359209+0000","last_became_peered":"2024-09-05T02:00:14.359209+0000","last_unstale":"2024-09-05T02:00:25.422874+0000","last_undegraded":"2024-09-05T02:00:25.422874+0000","last_fullsized":"2024-09-05T02:00:25.422874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:27.848906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59616,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":138,"num_read_kb":127,"num_write":238,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":169785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":478731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1117562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972387,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":482364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201320,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":478646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430251,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1075145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":473772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36908,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708244,"statfs":{"total":95995035648,"available":95957241856,"internally_reserved":0,"allocated":1601536,"data_stored":1070727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":477846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":473480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444101,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":528216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":475621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T02:01:13.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T02:01:13.948 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:14.339 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:14 smithi103 ceph-mon[30902]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:14.339 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:14 smithi103 ceph-mon[30902]: pgmap v310: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.5 KiB/s wr, 15 op/s 2024-09-05T02:01:14.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.416+0000 7f3ea4927640 1 -- 172.21.15.103:0/2022574739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102570 msgr2=0x7f3ea01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.416+0000 7f3ea4927640 1 --2- 172.21.15.103:0/2022574739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102570 0x7f3ea01029b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009900 tx=0x7f3e8802a480 comp rx=0 tx=0).stop 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.416+0000 7f3ea4927640 1 -- 172.21.15.103:0/2022574739 shutdown_connections 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.416+0000 7f3ea4927640 1 --2- 172.21.15.103:0/2022574739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea0102ef0 0x7f3ea010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.416+0000 7f3ea4927640 1 --2- 172.21.15.103:0/2022574739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102570 0x7f3ea01029b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.416+0000 7f3ea4927640 1 --2- 172.21.15.103:0/2022574739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ea0100e20 0x7f3ea0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.416+0000 7f3ea4927640 1 -- 172.21.15.103:0/2022574739 >> 172.21.15.103:0/2022574739 conn(0x7f3ea00fc7d0 msgr2=0x7f3ea00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.416+0000 7f3ea4927640 1 -- 172.21.15.103:0/2022574739 shutdown_connections 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.416+0000 7f3ea4927640 1 -- 172.21.15.103:0/2022574739 wait complete. 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.417+0000 7f3ea4927640 1 Processor -- start 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.417+0000 7f3ea4927640 1 -- start start 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.417+0000 7f3ea4927640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ea0100e20 0x7f3ea01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:14.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.417+0000 7f3ea4927640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea0102570 0x7f3ea0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.417+0000 7f3ea4927640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102ef0 0x7f3ea019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.417+0000 7f3ea4927640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0111f40 con 0x7f3ea0102ef0 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.417+0000 7f3ea4927640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0111dc0 con 0x7f3ea0102570 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.417+0000 7f3ea4927640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01120c0 con 0x7f3ea0100e20 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102ef0 0x7f3ea019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102ef0 0x7f3ea019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41724/0 (socket says 172.21.15.103:41724) 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9ed9d640 1 -- 172.21.15.103:0/1748952771 learned_addr learned my addr 172.21.15.103:0/1748952771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9e59c640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ea0100e20 0x7f3ea01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9dd9b640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea0102570 0x7f3ea0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9ed9d640 1 -- 172.21.15.103:0/1748952771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ea0100e20 msgr2=0x7f3ea01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9ed9d640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ea0100e20 0x7f3ea01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:14.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9ed9d640 1 -- 172.21.15.103:0/1748952771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea0102570 msgr2=0x7f3ea0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9ed9d640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea0102570 0x7f3ea0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9ed9d640 1 -- 172.21.15.103:0/1748952771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea019d370 con 0x7f3ea0102ef0 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e9ed9d640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102ef0 0x7f3ea019cc90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800cce0 tx=0x7f3e9800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e977fe640 1 -- 172.21.15.103:0/1748952771 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800f450 con 0x7f3ea0102ef0 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea019d660 con 0x7f3ea0102ef0 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e977fe640 1 -- 172.21.15.103:0/1748952771 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e9800faa0 con 0x7f3ea0102ef0 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3e977fe640 1 -- 172.21.15.103:0/1748952771 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98018b70 con 0x7f3ea0102ef0 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.418+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea01a4f40 con 0x7f3ea0102ef0 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.419+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea0101200 con 0x7f3ea0102ef0 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.421+0000 7f3e977fe640 1 -- 172.21.15.103:0/1748952771 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3e98018d10 con 0x7f3ea0102ef0 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.421+0000 7f3e977fe640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f3e8c0a2c10 0x7f3e8c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:14.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.421+0000 7f3e9e59c640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f3e8c0a2c10 0x7f3e8c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:14.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.421+0000 7f3e977fe640 1 -- 172.21.15.103:0/1748952771 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f3e980cf9d0 con 0x7f3ea0102ef0 2024-09-05T02:01:14.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.422+0000 7f3e9e59c640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f3e8c0a2c10 0x7f3e8c0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3e90005fd0 tx=0x7f3e90005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:14.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.424+0000 7f3e977fe640 1 -- 172.21.15.103:0/1748952771 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3e98014030 con 0x7f3ea0102ef0 2024-09-05T02:01:14.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.586+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ea0062630 con 0x7f3e8c0a2c10 2024-09-05T02:01:14.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.594+0000 7f3e977fe640 1 -- 172.21.15.103:0/1748952771 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340377 (secure 0 0 0) 0x7f3ea0062630 con 0x7f3e8c0a2c10 2024-09-05T02:01:14.599 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:14.601 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-05T02:01:14.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f3e8c0a2c10 msgr2=0x7f3e8c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:14.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f3e8c0a2c10 0x7f3e8c0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3e90005fd0 tx=0x7f3e90005850 comp rx=0 tx=0).stop 2024-09-05T02:01:14.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102ef0 msgr2=0x7f3ea019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:14.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102ef0 0x7f3ea019cc90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800cce0 tx=0x7f3e9800a620 comp rx=0 tx=0).stop 2024-09-05T02:01:14.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 shutdown_connections 2024-09-05T02:01:14.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea0102ef0 0x7f3ea019cc90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:14.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea0102570 0x7f3ea0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:14.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f3e8c0a2c10 0x7f3e8c0a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:14.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 --2- 172.21.15.103:0/1748952771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ea0100e20 0x7f3ea01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:14.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.599+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 >> 172.21.15.103:0/1748952771 conn(0x7f3ea00fc7d0 msgr2=0x7f3ea00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:14.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.600+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 shutdown_connections 2024-09-05T02:01:14.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:14.600+0000 7f3ea4927640 1 -- 172.21.15.103:0/1748952771 wait complete. 2024-09-05T02:01:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:14 smithi167 ceph-mon[35122]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:14 smithi167 ceph-mon[35122]: pgmap v310: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.5 KiB/s wr, 15 op/s 2024-09-05T02:01:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:14 smithi139 ceph-mon[34262]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:14 smithi139 ceph-mon[34262]: pgmap v310: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.5 KiB/s wr, 15 op/s 2024-09-05T02:01:15.236 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-05T02:01:13.095502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650218,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":269,"num_read_kb":236,"num_write":421,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391900,"kb_used_data":13532,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549924,"statfs":{"total":1151940427776,"available":1151539122176,"internally_reserved":0,"allocated":13856768,"data_stored":7659320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50844,"num_objects":14,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":126,"num_read_kb":112,"num_write":57,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002945"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:07.252880+0000","last_change":"2024-09-05T02:00:12.347638+0000","last_active":"2024-09-05T02:01:07.252880+0000","last_peered":"2024-09-05T02:01:07.252880+0000","last_clean":"2024-09-05T02:01:07.252880+0000","last_became_active":"2024-09-05T02:00:12.347314+0000","last_became_peered":"2024-09-05T02:00:12.347314+0000","last_unstale":"2024-09-05T02:01:07.252880+0000","last_undegraded":"2024-09-05T02:01:07.252880+0000","last_fullsized":"2024-09-05T02:01:07.252880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:08:39.084117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416951+0000","last_change":"2024-09-05T02:00:14.360262+0000","last_active":"2024-09-05T02:00:24.416951+0000","last_peered":"2024-09-05T02:00:24.416951+0000","last_clean":"2024-09-05T02:00:24.416951+0000","last_became_active":"2024-09-05T02:00:14.359965+0000","last_became_peered":"2024-09-05T02:00:14.359965+0000","last_unstale":"2024-09-05T02:00:24.416951+0000","last_undegraded":"2024-09-05T02:00:24.416951+0000","last_fullsized":"2024-09-05T02:00:24.416951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:01:56.109169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416298+0000","last_change":"2024-09-05T02:00:21.410692+0000","last_active":"2024-09-05T02:00:24.416298+0000","last_peered":"2024-09-05T02:00:24.416298+0000","last_clean":"2024-09-05T02:00:24.416298+0000","last_became_active":"2024-09-05T02:00:21.410298+0000","last_became_peered":"2024-09-05T02:00:21.410298+0000","last_unstale":"2024-09-05T02:00:24.416298+0000","last_undegraded":"2024-09-05T02:00:24.416298+0000","last_fullsized":"2024-09-05T02:00:24.416298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:25:31.241700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.417170+0000","last_change":"2024-09-05T02:00:12.783109+0000","last_active":"2024-09-05T02:00:24.417170+0000","last_peered":"2024-09-05T02:00:24.417170+0000","last_clean":"2024-09-05T02:00:24.417170+0000","last_became_active":"2024-09-05T02:00:12.782773+0000","last_became_peered":"2024-09-05T02:00:12.782773+0000","last_unstale":"2024-09-05T02:00:24.417170+0000","last_undegraded":"2024-09-05T02:00:24.417170+0000","last_fullsized":"2024-09-05T02:00:24.417170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:29:12.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421317+0000","last_change":"2024-09-05T02:00:14.370425+0000","last_active":"2024-09-05T02:00:25.421317+0000","last_peered":"2024-09-05T02:00:25.421317+0000","last_clean":"2024-09-05T02:00:25.421317+0000","last_became_active":"2024-09-05T02:00:14.370102+0000","last_became_peered":"2024-09-05T02:00:14.370102+0000","last_unstale":"2024-09-05T02:00:25.421317+0000","last_undegraded":"2024-09-05T02:00:25.421317+0000","last_fullsized":"2024-09-05T02:00:25.421317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:22:43.837418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422396+0000","last_change":"2024-09-05T02:00:21.413082+0000","last_active":"2024-09-05T02:00:25.422396+0000","last_peered":"2024-09-05T02:00:25.422396+0000","last_clean":"2024-09-05T02:00:25.422396+0000","last_became_active":"2024-09-05T02:00:21.412661+0000","last_became_peered":"2024-09-05T02:00:21.412661+0000","last_unstale":"2024-09-05T02:00:25.422396+0000","last_undegraded":"2024-09-05T02:00:25.422396+0000","last_fullsized":"2024-09-05T02:00:25.422396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:51:41.225573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:59.240594+0000","last_change":"2024-09-05T02:00:12.353894+0000","last_active":"2024-09-05T02:00:59.240594+0000","last_peered":"2024-09-05T02:00:59.240594+0000","last_clean":"2024-09-05T02:00:59.240594+0000","last_became_active":"2024-09-05T02:00:12.353483+0000","last_became_peered":"2024-09-05T02:00:12.353483+0000","last_unstale":"2024-09-05T02:00:59.240594+0000","last_undegraded":"2024-09-05T02:00:59.240594+0000","last_fullsized":"2024-09-05T02:00:59.240594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:42.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421480+0000","last_change":"2024-09-05T02:00:14.356441+0000","last_active":"2024-09-05T02:00:25.421480+0000","last_peered":"2024-09-05T02:00:25.421480+0000","last_clean":"2024-09-05T02:00:25.421480+0000","last_became_active":"2024-09-05T02:00:14.356092+0000","last_became_peered":"2024-09-05T02:00:14.356092+0000","last_unstale":"2024-09-05T02:00:25.421480+0000","last_undegraded":"2024-09-05T02:00:25.421480+0000","last_fullsized":"2024-09-05T02:00:25.421480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:33:55.416968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423573+0000","last_change":"2024-09-05T02:00:21.418324+0000","last_active":"2024-09-05T02:00:25.423573+0000","last_peered":"2024-09-05T02:00:25.423573+0000","last_clean":"2024-09-05T02:00:25.423573+0000","last_became_active":"2024-09-05T02:00:21.417934+0000","last_became_peered":"2024-09-05T02:00:21.417934+0000","last_unstale":"2024-09-05T02:00:25.423573+0000","last_undegraded":"2024-09-05T02:00:25.423573+0000","last_fullsized":"2024-09-05T02:00:25.423573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:50.425823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.230762+0000","last_change":"2024-09-05T02:00:12.348637+0000","last_active":"2024-09-05T02:00:58.230762+0000","last_peered":"2024-09-05T02:00:58.230762+0000","last_clean":"2024-09-05T02:00:58.230762+0000","last_became_active":"2024-09-05T02:00:12.348287+0000","last_became_peered":"2024-09-05T02:00:12.348287+0000","last_unstale":"2024-09-05T02:00:58.230762+0000","last_undegraded":"2024-09-05T02:00:58.230762+0000","last_fullsized":"2024-09-05T02:00:58.230762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:52:58.368026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.691044+0000","last_change":"2024-09-05T02:00:14.368414+0000","last_active":"2024-09-05T02:00:24.691044+0000","last_peered":"2024-09-05T02:00:24.691044+0000","last_clean":"2024-09-05T02:00:24.691044+0000","last_became_active":"2024-09-05T02:00:14.368018+0000","last_became_peered":"2024-09-05T02:00:14.368018+0000","last_unstale":"2024-09-05T02:00:24.691044+0000","last_undegraded":"2024-09-05T02:00:24.691044+0000","last_fullsized":"2024-09-05T02:00:24.691044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:08.246420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423072+0000","last_change":"2024-09-05T02:00:21.416649+0000","last_active":"2024-09-05T02:00:25.423072+0000","last_peered":"2024-09-05T02:00:25.423072+0000","last_clean":"2024-09-05T02:00:25.423072+0000","last_became_active":"2024-09-05T02:00:21.416247+0000","last_became_peered":"2024-09-05T02:00:21.416247+0000","last_unstale":"2024-09-05T02:00:25.423072+0000","last_undegraded":"2024-09-05T02:00:25.423072+0000","last_fullsized":"2024-09-05T02:00:25.423072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:10:00.115655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421769+0000","last_change":"2024-09-05T02:00:14.357719+0000","last_active":"2024-09-05T02:00:25.421769+0000","last_peered":"2024-09-05T02:00:25.421769+0000","last_clean":"2024-09-05T02:00:25.421769+0000","last_became_active":"2024-09-05T02:00:14.357211+0000","last_became_peered":"2024-09-05T02:00:14.357211+0000","last_unstale":"2024-09-05T02:00:25.421769+0000","last_undegraded":"2024-09-05T02:00:25.421769+0000","last_fullsized":"2024-09-05T02:00:25.421769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:27:15.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:55.202216+0000","last_change":"2024-09-05T02:00:12.353893+0000","last_active":"2024-09-05T02:00:55.202216+0000","last_peered":"2024-09-05T02:00:55.202216+0000","last_clean":"2024-09-05T02:00:55.202216+0000","last_became_active":"2024-09-05T02:00:12.353531+0000","last_became_peered":"2024-09-05T02:00:12.353531+0000","last_unstale":"2024-09-05T02:00:55.202216+0000","last_undegraded":"2024-09-05T02:00:55.202216+0000","last_fullsized":"2024-09-05T02:00:55.202216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:58.597446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423504+0000","last_change":"2024-09-05T02:00:21.415203+0000","last_active":"2024-09-05T02:00:25.423504+0000","last_peered":"2024-09-05T02:00:25.423504+0000","last_clean":"2024-09-05T02:00:25.423504+0000","last_became_active":"2024-09-05T02:00:21.414767+0000","last_became_peered":"2024-09-05T02:00:21.414767+0000","last_unstale":"2024-09-05T02:00:25.423504+0000","last_undegraded":"2024-09-05T02:00:25.423504+0000","last_fullsized":"2024-09-05T02:00:25.423504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:01:48.398448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416709+0000","last_change":"2024-09-05T02:00:14.369417+0000","last_active":"2024-09-05T02:00:24.416709+0000","last_peered":"2024-09-05T02:00:24.416709+0000","last_clean":"2024-09-05T02:00:24.416709+0000","last_became_active":"2024-09-05T02:00:14.369093+0000","last_became_peered":"2024-09-05T02:00:14.369093+0000","last_unstale":"2024-09-05T02:00:24.416709+0000","last_undegraded":"2024-09-05T02:00:24.416709+0000","last_fullsized":"2024-09-05T02:00:24.416709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:01:34.239300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:57.216177+0000","last_change":"2024-09-05T02:00:12.347945+0000","last_active":"2024-09-05T02:00:57.216177+0000","last_peered":"2024-09-05T02:00:57.216177+0000","last_clean":"2024-09-05T02:00:57.216177+0000","last_became_active":"2024-09-05T02:00:12.347577+0000","last_became_peered":"2024-09-05T02:00:12.347577+0000","last_unstale":"2024-09-05T02:00:57.216177+0000","last_undegraded":"2024-09-05T02:00:57.216177+0000","last_fullsized":"2024-09-05T02:00:57.216177+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:53:23.947159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423153+0000","last_change":"2024-09-05T02:00:21.419070+0000","last_active":"2024-09-05T02:00:25.423153+0000","last_peered":"2024-09-05T02:00:25.423153+0000","last_clean":"2024-09-05T02:00:25.423153+0000","last_became_active":"2024-09-05T02:00:21.418315+0000","last_became_peered":"2024-09-05T02:00:21.418315+0000","last_unstale":"2024-09-05T02:00:25.423153+0000","last_undegraded":"2024-09-05T02:00:25.423153+0000","last_fullsized":"2024-09-05T02:00:25.423153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:17.974047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422211+0000","last_change":"2024-09-05T02:00:14.362965+0000","last_active":"2024-09-05T02:00:25.422211+0000","last_peered":"2024-09-05T02:00:25.422211+0000","last_clean":"2024-09-05T02:00:25.422211+0000","last_became_active":"2024-09-05T02:00:14.361842+0000","last_became_peered":"2024-09-05T02:00:14.361842+0000","last_unstale":"2024-09-05T02:00:25.422211+0000","last_undegraded":"2024-09-05T02:00:25.422211+0000","last_fullsized":"2024-09-05T02:00:25.422211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:45:10.565392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.227725+0000","last_change":"2024-09-05T02:00:12.349368+0000","last_active":"2024-09-05T02:00:58.227725+0000","last_peered":"2024-09-05T02:00:58.227725+0000","last_clean":"2024-09-05T02:00:58.227725+0000","last_became_active":"2024-09-05T02:00:12.348990+0000","last_became_peered":"2024-09-05T02:00:12.348990+0000","last_unstale":"2024-09-05T02:00:58.227725+0000","last_undegraded":"2024-09-05T02:00:58.227725+0000","last_fullsized":"2024-09-05T02:00:58.227725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:45:04.998316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422304+0000","last_change":"2024-09-05T02:00:21.415467+0000","last_active":"2024-09-05T02:00:25.422304+0000","last_peered":"2024-09-05T02:00:25.422304+0000","last_clean":"2024-09-05T02:00:25.422304+0000","last_became_active":"2024-09-05T02:00:21.415138+0000","last_became_peered":"2024-09-05T02:00:21.415138+0000","last_unstale":"2024-09-05T02:00:25.422304+0000","last_undegraded":"2024-09-05T02:00:25.422304+0000","last_fullsized":"2024-09-05T02:00:25.422304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:51.221383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421232+0000","last_change":"2024-09-05T02:00:14.357961+0000","last_active":"2024-09-05T02:00:25.421232+0000","last_peered":"2024-09-05T02:00:25.421232+0000","last_clean":"2024-09-05T02:00:25.421232+0000","last_became_active":"2024-09-05T02:00:14.357624+0000","last_became_peered":"2024-09-05T02:00:14.357624+0000","last_unstale":"2024-09-05T02:00:25.421232+0000","last_undegraded":"2024-09-05T02:00:25.421232+0000","last_fullsized":"2024-09-05T02:00:25.421232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:01.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:55.202613+0000","last_change":"2024-09-05T02:00:12.783304+0000","last_active":"2024-09-05T02:00:55.202613+0000","last_peered":"2024-09-05T02:00:55.202613+0000","last_clean":"2024-09-05T02:00:55.202613+0000","last_became_active":"2024-09-05T02:00:12.782953+0000","last_became_peered":"2024-09-05T02:00:12.782953+0000","last_unstale":"2024-09-05T02:00:55.202613+0000","last_undegraded":"2024-09-05T02:00:55.202613+0000","last_fullsized":"2024-09-05T02:00:55.202613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:50:59.130705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423575+0000","last_change":"2024-09-05T02:00:21.418357+0000","last_active":"2024-09-05T02:00:25.423575+0000","last_peered":"2024-09-05T02:00:25.423575+0000","last_clean":"2024-09-05T02:00:25.423575+0000","last_became_active":"2024-09-05T02:00:21.417779+0000","last_became_peered":"2024-09-05T02:00:21.417779+0000","last_unstale":"2024-09-05T02:00:25.423575+0000","last_undegraded":"2024-09-05T02:00:25.423575+0000","last_fullsized":"2024-09-05T02:00:25.423575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:37:45.423447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421992+0000","last_change":"2024-09-05T02:00:14.362857+0000","last_active":"2024-09-05T02:00:25.421992+0000","last_peered":"2024-09-05T02:00:25.421992+0000","last_clean":"2024-09-05T02:00:25.421992+0000","last_became_active":"2024-09-05T02:00:14.362556+0000","last_became_peered":"2024-09-05T02:00:14.362556+0000","last_unstale":"2024-09-05T02:00:25.421992+0000","last_undegraded":"2024-09-05T02:00:25.421992+0000","last_fullsized":"2024-09-05T02:00:25.421992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:44:03.413055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.228902+0000","last_change":"2024-09-05T02:00:12.512747+0000","last_active":"2024-09-05T02:00:58.228902+0000","last_peered":"2024-09-05T02:00:58.228902+0000","last_clean":"2024-09-05T02:00:58.228902+0000","last_became_active":"2024-09-05T02:00:12.512471+0000","last_became_peered":"2024-09-05T02:00:12.512471+0000","last_unstale":"2024-09-05T02:00:58.228902+0000","last_undegraded":"2024-09-05T02:00:58.228902+0000","last_fullsized":"2024-09-05T02:00:58.228902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:50.235020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416329+0000","last_change":"2024-09-05T02:00:21.413947+0000","last_active":"2024-09-05T02:00:24.416329+0000","last_peered":"2024-09-05T02:00:24.416329+0000","last_clean":"2024-09-05T02:00:24.416329+0000","last_became_active":"2024-09-05T02:00:21.413488+0000","last_became_peered":"2024-09-05T02:00:21.413488+0000","last_unstale":"2024-09-05T02:00:24.416329+0000","last_undegraded":"2024-09-05T02:00:24.416329+0000","last_fullsized":"2024-09-05T02:00:24.416329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:52:16.201325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.417065+0000","last_change":"2024-09-05T02:00:14.367009+0000","last_active":"2024-09-05T02:00:24.417065+0000","last_peered":"2024-09-05T02:00:24.417065+0000","last_clean":"2024-09-05T02:00:24.417065+0000","last_became_active":"2024-09-05T02:00:14.366698+0000","last_became_peered":"2024-09-05T02:00:14.366698+0000","last_unstale":"2024-09-05T02:00:24.417065+0000","last_undegraded":"2024-09-05T02:00:24.417065+0000","last_fullsized":"2024-09-05T02:00:24.417065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:37:43.811300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:06.380715+0000","last_change":"2024-09-05T02:00:12.346918+0000","last_active":"2024-09-05T02:01:06.380715+0000","last_peered":"2024-09-05T02:01:06.380715+0000","last_clean":"2024-09-05T02:01:06.380715+0000","last_became_active":"2024-09-05T02:00:12.346541+0000","last_became_peered":"2024-09-05T02:00:12.346541+0000","last_unstale":"2024-09-05T02:01:06.380715+0000","last_undegraded":"2024-09-05T02:01:06.380715+0000","last_fullsized":"2024-09-05T02:01:06.380715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:50.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416503+0000","last_change":"2024-09-05T02:00:21.413077+0000","last_active":"2024-09-05T02:00:24.416503+0000","last_peered":"2024-09-05T02:00:24.416503+0000","last_clean":"2024-09-05T02:00:24.416503+0000","last_became_active":"2024-09-05T02:00:21.412698+0000","last_became_peered":"2024-09-05T02:00:21.412698+0000","last_unstale":"2024-09-05T02:00:24.416503+0000","last_undegraded":"2024-09-05T02:00:24.416503+0000","last_fullsized":"2024-09-05T02:00:24.416503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:12:18.463614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415862+0000","last_change":"2024-09-05T02:00:14.363072+0000","last_active":"2024-09-05T02:00:24.415862+0000","last_peered":"2024-09-05T02:00:24.415862+0000","last_clean":"2024-09-05T02:00:24.415862+0000","last_became_active":"2024-09-05T02:00:14.361932+0000","last_became_peered":"2024-09-05T02:00:14.361932+0000","last_unstale":"2024-09-05T02:00:24.415862+0000","last_undegraded":"2024-09-05T02:00:24.415862+0000","last_fullsized":"2024-09-05T02:00:24.415862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:20:13.080963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.229875+0000","last_change":"2024-09-05T02:00:12.783825+0000","last_active":"2024-09-05T02:00:58.229875+0000","last_peered":"2024-09-05T02:00:58.229875+0000","last_clean":"2024-09-05T02:00:58.229875+0000","last_became_active":"2024-09-05T02:00:12.783298+0000","last_became_peered":"2024-09-05T02:00:12.783298+0000","last_unstale":"2024-09-05T02:00:58.229875+0000","last_undegraded":"2024-09-05T02:00:58.229875+0000","last_fullsized":"2024-09-05T02:00:58.229875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:00:10.986318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422188+0000","last_change":"2024-09-05T02:00:21.416474+0000","last_active":"2024-09-05T02:00:25.422188+0000","last_peered":"2024-09-05T02:00:25.422188+0000","last_clean":"2024-09-05T02:00:25.422188+0000","last_became_active":"2024-09-05T02:00:21.416006+0000","last_became_peered":"2024-09-05T02:00:21.416006+0000","last_unstale":"2024-09-05T02:00:25.422188+0000","last_undegraded":"2024-09-05T02:00:25.422188+0000","last_fullsized":"2024-09-05T02:00:25.422188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:00:18.587895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.613698+0000","last_change":"2024-09-05T02:00:14.362458+0000","last_active":"2024-09-05T02:00:24.613698+0000","last_peered":"2024-09-05T02:00:24.613698+0000","last_clean":"2024-09-05T02:00:24.613698+0000","last_became_active":"2024-09-05T02:00:14.362080+0000","last_became_peered":"2024-09-05T02:00:14.362080+0000","last_unstale":"2024-09-05T02:00:24.613698+0000","last_undegraded":"2024-09-05T02:00:24.613698+0000","last_fullsized":"2024-09-05T02:00:24.613698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:47:49.498468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421986+0000","last_change":"2024-09-05T02:00:12.783445+0000","last_active":"2024-09-05T02:00:25.421986+0000","last_peered":"2024-09-05T02:00:25.421986+0000","last_clean":"2024-09-05T02:00:25.421986+0000","last_became_active":"2024-09-05T02:00:12.783131+0000","last_became_peered":"2024-09-05T02:00:12.783131+0000","last_unstale":"2024-09-05T02:00:25.421986+0000","last_undegraded":"2024-09-05T02:00:25.421986+0000","last_fullsized":"2024-09-05T02:00:25.421986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:21:57.743629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421405+0000","last_change":"2024-09-05T02:00:21.418568+0000","last_active":"2024-09-05T02:00:25.421405+0000","last_peered":"2024-09-05T02:00:25.421405+0000","last_clean":"2024-09-05T02:00:25.421405+0000","last_became_active":"2024-09-05T02:00:21.418152+0000","last_became_peered":"2024-09-05T02:00:21.418152+0000","last_unstale":"2024-09-05T02:00:25.421405+0000","last_undegraded":"2024-09-05T02:00:25.421405+0000","last_fullsized":"2024-09-05T02:00:25.421405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:50:17.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.613582+0000","last_change":"2024-09-05T02:00:14.364556+0000","last_active":"2024-09-05T02:00:24.613582+0000","last_peered":"2024-09-05T02:00:24.613582+0000","last_clean":"2024-09-05T02:00:24.613582+0000","last_became_active":"2024-09-05T02:00:14.364285+0000","last_became_peered":"2024-09-05T02:00:14.364285+0000","last_unstale":"2024-09-05T02:00:24.613582+0000","last_undegraded":"2024-09-05T02:00:24.613582+0000","last_fullsized":"2024-09-05T02:00:24.613582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:24:54.901510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:10.256963+0000","last_change":"2024-09-05T02:00:12.347993+0000","last_active":"2024-09-05T02:01:10.256963+0000","last_peered":"2024-09-05T02:01:10.256963+0000","last_clean":"2024-09-05T02:01:10.256963+0000","last_became_active":"2024-09-05T02:00:12.347656+0000","last_became_peered":"2024-09-05T02:00:12.347656+0000","last_unstale":"2024-09-05T02:01:10.256963+0000","last_undegraded":"2024-09-05T02:01:10.256963+0000","last_fullsized":"2024-09-05T02:01:10.256963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:31:23.820379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415609+0000","last_change":"2024-09-05T02:00:21.410786+0000","last_active":"2024-09-05T02:00:24.415609+0000","last_peered":"2024-09-05T02:00:24.415609+0000","last_clean":"2024-09-05T02:00:24.415609+0000","last_became_active":"2024-09-05T02:00:21.410440+0000","last_became_peered":"2024-09-05T02:00:21.410440+0000","last_unstale":"2024-09-05T02:00:24.415609+0000","last_undegraded":"2024-09-05T02:00:24.415609+0000","last_fullsized":"2024-09-05T02:00:24.415609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:22:14.644413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690706+0000","last_change":"2024-09-05T02:00:14.357893+0000","last_active":"2024-09-05T02:00:24.690706+0000","last_peered":"2024-09-05T02:00:24.690706+0000","last_clean":"2024-09-05T02:00:24.690706+0000","last_became_active":"2024-09-05T02:00:14.357551+0000","last_became_peered":"2024-09-05T02:00:14.357551+0000","last_unstale":"2024-09-05T02:00:24.690706+0000","last_undegraded":"2024-09-05T02:00:24.690706+0000","last_fullsized":"2024-09-05T02:00:24.690706+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:04:15.466460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:07.258092+0000","last_change":"2024-09-05T02:00:12.783358+0000","last_active":"2024-09-05T02:01:07.258092+0000","last_peered":"2024-09-05T02:01:07.258092+0000","last_clean":"2024-09-05T02:01:07.258092+0000","last_became_active":"2024-09-05T02:00:12.782950+0000","last_became_peered":"2024-09-05T02:00:12.782950+0000","last_unstale":"2024-09-05T02:01:07.258092+0000","last_undegraded":"2024-09-05T02:01:07.258092+0000","last_fullsized":"2024-09-05T02:01:07.258092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:17:38.823019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415727+0000","last_change":"2024-09-05T02:00:21.416389+0000","last_active":"2024-09-05T02:00:24.415727+0000","last_peered":"2024-09-05T02:00:24.415727+0000","last_clean":"2024-09-05T02:00:24.415727+0000","last_became_active":"2024-09-05T02:00:21.416065+0000","last_became_peered":"2024-09-05T02:00:21.416065+0000","last_unstale":"2024-09-05T02:00:24.415727+0000","last_undegraded":"2024-09-05T02:00:24.415727+0000","last_fullsized":"2024-09-05T02:00:24.415727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:15.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415274+0000","last_change":"2024-09-05T02:00:14.361404+0000","last_active":"2024-09-05T02:00:24.415274+0000","last_peered":"2024-09-05T02:00:24.415274+0000","last_clean":"2024-09-05T02:00:24.415274+0000","last_became_active":"2024-09-05T02:00:14.361031+0000","last_became_peered":"2024-09-05T02:00:14.361031+0000","last_unstale":"2024-09-05T02:00:24.415274+0000","last_undegraded":"2024-09-05T02:00:24.415274+0000","last_fullsized":"2024-09-05T02:00:24.415274+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:40.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:56.206346+0000","last_change":"2024-09-05T02:00:12.351656+0000","last_active":"2024-09-05T02:00:56.206346+0000","last_peered":"2024-09-05T02:00:56.206346+0000","last_clean":"2024-09-05T02:00:56.206346+0000","last_became_active":"2024-09-05T02:00:12.351310+0000","last_became_peered":"2024-09-05T02:00:12.351310+0000","last_unstale":"2024-09-05T02:00:56.206346+0000","last_undegraded":"2024-09-05T02:00:56.206346+0000","last_fullsized":"2024-09-05T02:00:56.206346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:55:27.949471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.691213+0000","last_change":"2024-09-05T02:00:21.417300+0000","last_active":"2024-09-05T02:00:24.691213+0000","last_peered":"2024-09-05T02:00:24.691213+0000","last_clean":"2024-09-05T02:00:24.691213+0000","last_became_active":"2024-09-05T02:00:21.415401+0000","last_became_peered":"2024-09-05T02:00:21.415401+0000","last_unstale":"2024-09-05T02:00:24.691213+0000","last_undegraded":"2024-09-05T02:00:24.691213+0000","last_fullsized":"2024-09-05T02:00:24.691213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:44:02.874857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416841+0000","last_change":"2024-09-05T02:00:14.359356+0000","last_active":"2024-09-05T02:00:24.416841+0000","last_peered":"2024-09-05T02:00:24.416841+0000","last_clean":"2024-09-05T02:00:24.416841+0000","last_became_active":"2024-09-05T02:00:14.359078+0000","last_became_peered":"2024-09-05T02:00:14.359078+0000","last_unstale":"2024-09-05T02:00:24.416841+0000","last_undegraded":"2024-09-05T02:00:24.416841+0000","last_fullsized":"2024-09-05T02:00:24.416841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:40:24.219962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:59.240636+0000","last_change":"2024-09-05T02:00:12.349749+0000","last_active":"2024-09-05T02:00:59.240636+0000","last_peered":"2024-09-05T02:00:59.240636+0000","last_clean":"2024-09-05T02:00:59.240636+0000","last_became_active":"2024-09-05T02:00:12.349247+0000","last_became_peered":"2024-09-05T02:00:12.349247+0000","last_unstale":"2024-09-05T02:00:59.240636+0000","last_undegraded":"2024-09-05T02:00:59.240636+0000","last_fullsized":"2024-09-05T02:00:59.240636+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:04:26.361209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":343,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422612+0000","last_change":"2024-09-05T01:58:57.552989+0000","last_active":"2024-09-05T02:00:25.422612+0000","last_peered":"2024-09-05T02:00:25.422612+0000","last_clean":"2024-09-05T02:00:25.422612+0000","last_became_active":"2024-09-05T01:58:57.552511+0000","last_became_peered":"2024-09-05T01:58:57.552511+0000","last_unstale":"2024-09-05T02:00:25.422612+0000","last_undegraded":"2024-09-05T02:00:25.422612+0000","last_fullsized":"2024-09-05T02:00:25.422612+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_clean_scrub_stamp":"2024-09-05T01:55:35.678445+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:53.711210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422392+0000","last_change":"2024-09-05T02:00:21.421456+0000","last_active":"2024-09-05T02:00:25.422392+0000","last_peered":"2024-09-05T02:00:25.422392+0000","last_clean":"2024-09-05T02:00:25.422392+0000","last_became_active":"2024-09-05T02:00:21.421074+0000","last_became_peered":"2024-09-05T02:00:21.421074+0000","last_unstale":"2024-09-05T02:00:25.422392+0000","last_undegraded":"2024-09-05T02:00:25.422392+0000","last_fullsized":"2024-09-05T02:00:25.422392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:46:00.660079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422568+0000","last_change":"2024-09-05T02:00:14.357638+0000","last_active":"2024-09-05T02:00:25.422568+0000","last_peered":"2024-09-05T02:00:25.422568+0000","last_clean":"2024-09-05T02:00:25.422568+0000","last_became_active":"2024-09-05T02:00:14.357125+0000","last_became_peered":"2024-09-05T02:00:14.357125+0000","last_unstale":"2024-09-05T02:00:25.422568+0000","last_undegraded":"2024-09-05T02:00:25.422568+0000","last_fullsized":"2024-09-05T02:00:25.422568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:02:04.343235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:08.258835+0000","last_change":"2024-09-05T02:00:12.783627+0000","last_active":"2024-09-05T02:01:08.258835+0000","last_peered":"2024-09-05T02:01:08.258835+0000","last_clean":"2024-09-05T02:01:08.258835+0000","last_became_active":"2024-09-05T02:00:12.783112+0000","last_became_peered":"2024-09-05T02:00:12.783112+0000","last_unstale":"2024-09-05T02:01:08.258835+0000","last_undegraded":"2024-09-05T02:01:08.258835+0000","last_fullsized":"2024-09-05T02:01:08.258835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:39:45.190311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421455+0000","last_change":"2024-09-05T02:00:21.413069+0000","last_active":"2024-09-05T02:00:25.421455+0000","last_peered":"2024-09-05T02:00:25.421455+0000","last_clean":"2024-09-05T02:00:25.421455+0000","last_became_active":"2024-09-05T02:00:21.412687+0000","last_became_peered":"2024-09-05T02:00:21.412687+0000","last_unstale":"2024-09-05T02:00:25.421455+0000","last_undegraded":"2024-09-05T02:00:25.421455+0000","last_fullsized":"2024-09-05T02:00:25.421455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:28.883071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421766+0000","last_change":"2024-09-05T02:00:14.363197+0000","last_active":"2024-09-05T02:00:25.421766+0000","last_peered":"2024-09-05T02:00:25.421766+0000","last_clean":"2024-09-05T02:00:25.421766+0000","last_became_active":"2024-09-05T02:00:14.362854+0000","last_became_peered":"2024-09-05T02:00:14.362854+0000","last_unstale":"2024-09-05T02:00:25.421766+0000","last_undegraded":"2024-09-05T02:00:25.421766+0000","last_fullsized":"2024-09-05T02:00:25.421766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:29:28.770604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.231039+0000","last_change":"2024-09-05T02:00:12.348715+0000","last_active":"2024-09-05T02:00:58.231039+0000","last_peered":"2024-09-05T02:00:58.231039+0000","last_clean":"2024-09-05T02:00:58.231039+0000","last_became_active":"2024-09-05T02:00:12.348322+0000","last_became_peered":"2024-09-05T02:00:12.348322+0000","last_unstale":"2024-09-05T02:00:58.231039+0000","last_undegraded":"2024-09-05T02:00:58.231039+0000","last_fullsized":"2024-09-05T02:00:58.231039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:33:55.551080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421146+0000","last_change":"2024-09-05T02:00:21.414103+0000","last_active":"2024-09-05T02:00:25.421146+0000","last_peered":"2024-09-05T02:00:25.421146+0000","last_clean":"2024-09-05T02:00:25.421146+0000","last_became_active":"2024-09-05T02:00:21.413706+0000","last_became_peered":"2024-09-05T02:00:21.413706+0000","last_unstale":"2024-09-05T02:00:25.421146+0000","last_undegraded":"2024-09-05T02:00:25.421146+0000","last_fullsized":"2024-09-05T02:00:25.421146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:46:41.226053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421868+0000","last_change":"2024-09-05T02:00:14.355969+0000","last_active":"2024-09-05T02:00:25.421868+0000","last_peered":"2024-09-05T02:00:25.421868+0000","last_clean":"2024-09-05T02:00:25.421868+0000","last_became_active":"2024-09-05T02:00:14.355581+0000","last_became_peered":"2024-09-05T02:00:14.355581+0000","last_unstale":"2024-09-05T02:00:25.421868+0000","last_undegraded":"2024-09-05T02:00:25.421868+0000","last_fullsized":"2024-09-05T02:00:25.421868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:55:22.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.229946+0000","last_change":"2024-09-05T02:00:12.348148+0000","last_active":"2024-09-05T02:00:58.229946+0000","last_peered":"2024-09-05T02:00:58.229946+0000","last_clean":"2024-09-05T02:00:58.229946+0000","last_became_active":"2024-09-05T02:00:12.347857+0000","last_became_peered":"2024-09-05T02:00:12.347857+0000","last_unstale":"2024-09-05T02:00:58.229946+0000","last_undegraded":"2024-09-05T02:00:58.229946+0000","last_fullsized":"2024-09-05T02:00:58.229946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:22:21.368082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690558+0000","last_change":"2024-09-05T02:00:21.417476+0000","last_active":"2024-09-05T02:00:24.690558+0000","last_peered":"2024-09-05T02:00:24.690558+0000","last_clean":"2024-09-05T02:00:24.690558+0000","last_became_active":"2024-09-05T02:00:21.417036+0000","last_became_peered":"2024-09-05T02:00:21.417036+0000","last_unstale":"2024-09-05T02:00:24.690558+0000","last_undegraded":"2024-09-05T02:00:24.690558+0000","last_fullsized":"2024-09-05T02:00:24.690558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:15:22.510948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422449+0000","last_change":"2024-09-05T02:00:14.358542+0000","last_active":"2024-09-05T02:00:25.422449+0000","last_peered":"2024-09-05T02:00:25.422449+0000","last_clean":"2024-09-05T02:00:25.422449+0000","last_became_active":"2024-09-05T02:00:14.358228+0000","last_became_peered":"2024-09-05T02:00:14.358228+0000","last_unstale":"2024-09-05T02:00:25.422449+0000","last_undegraded":"2024-09-05T02:00:25.422449+0000","last_fullsized":"2024-09-05T02:00:25.422449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:01.627453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.202513+0000","last_change":"2024-09-05T02:00:12.345887+0000","last_active":"2024-09-05T02:00:58.202513+0000","last_peered":"2024-09-05T02:00:58.202513+0000","last_clean":"2024-09-05T02:00:58.202513+0000","last_became_active":"2024-09-05T02:00:12.345491+0000","last_became_peered":"2024-09-05T02:00:12.345491+0000","last_unstale":"2024-09-05T02:00:58.202513+0000","last_undegraded":"2024-09-05T02:00:58.202513+0000","last_fullsized":"2024-09-05T02:00:58.202513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:10:32.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690649+0000","last_change":"2024-09-05T02:00:21.418210+0000","last_active":"2024-09-05T02:00:24.690649+0000","last_peered":"2024-09-05T02:00:24.690649+0000","last_clean":"2024-09-05T02:00:24.690649+0000","last_became_active":"2024-09-05T02:00:21.417769+0000","last_became_peered":"2024-09-05T02:00:21.417769+0000","last_unstale":"2024-09-05T02:00:24.690649+0000","last_undegraded":"2024-09-05T02:00:24.690649+0000","last_fullsized":"2024-09-05T02:00:24.690649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:05.050499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690593+0000","last_change":"2024-09-05T02:00:14.367555+0000","last_active":"2024-09-05T02:00:24.690593+0000","last_peered":"2024-09-05T02:00:24.690593+0000","last_clean":"2024-09-05T02:00:24.690593+0000","last_became_active":"2024-09-05T02:00:14.367222+0000","last_became_peered":"2024-09-05T02:00:14.367222+0000","last_unstale":"2024-09-05T02:00:24.690593+0000","last_undegraded":"2024-09-05T02:00:24.690593+0000","last_fullsized":"2024-09-05T02:00:24.690593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:04:26.172141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.230148+0000","last_change":"2024-09-05T02:00:12.785867+0000","last_active":"2024-09-05T02:00:58.230148+0000","last_peered":"2024-09-05T02:00:58.230148+0000","last_clean":"2024-09-05T02:00:58.230148+0000","last_became_active":"2024-09-05T02:00:12.785470+0000","last_became_peered":"2024-09-05T02:00:12.785470+0000","last_unstale":"2024-09-05T02:00:58.230148+0000","last_undegraded":"2024-09-05T02:00:58.230148+0000","last_fullsized":"2024-09-05T02:00:58.230148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:57:29.367257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415933+0000","last_change":"2024-09-05T02:00:21.412374+0000","last_active":"2024-09-05T02:00:24.415933+0000","last_peered":"2024-09-05T02:00:24.415933+0000","last_clean":"2024-09-05T02:00:24.415933+0000","last_became_active":"2024-09-05T02:00:21.411926+0000","last_became_peered":"2024-09-05T02:00:21.411926+0000","last_unstale":"2024-09-05T02:00:24.415933+0000","last_undegraded":"2024-09-05T02:00:24.415933+0000","last_fullsized":"2024-09-05T02:00:24.415933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:38:29.657795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422370+0000","last_change":"2024-09-05T02:00:14.361332+0000","last_active":"2024-09-05T02:00:25.422370+0000","last_peered":"2024-09-05T02:00:25.422370+0000","last_clean":"2024-09-05T02:00:25.422370+0000","last_became_active":"2024-09-05T02:00:14.360995+0000","last_became_peered":"2024-09-05T02:00:14.360995+0000","last_unstale":"2024-09-05T02:00:25.422370+0000","last_undegraded":"2024-09-05T02:00:25.422370+0000","last_fullsized":"2024-09-05T02:00:25.422370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:12:11.104876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:07.258582+0000","last_change":"2024-09-05T02:00:12.349175+0000","last_active":"2024-09-05T02:01:07.258582+0000","last_peered":"2024-09-05T02:01:07.258582+0000","last_clean":"2024-09-05T02:01:07.258582+0000","last_became_active":"2024-09-05T02:00:12.348647+0000","last_became_peered":"2024-09-05T02:00:12.348647+0000","last_unstale":"2024-09-05T02:01:07.258582+0000","last_undegraded":"2024-09-05T02:01:07.258582+0000","last_fullsized":"2024-09-05T02:01:07.258582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:38:13.363462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422455+0000","last_change":"2024-09-05T02:00:21.414252+0000","last_active":"2024-09-05T02:00:25.422455+0000","last_peered":"2024-09-05T02:00:25.422455+0000","last_clean":"2024-09-05T02:00:25.422455+0000","last_became_active":"2024-09-05T02:00:21.412483+0000","last_became_peered":"2024-09-05T02:00:21.412483+0000","last_unstale":"2024-09-05T02:00:25.422455+0000","last_undegraded":"2024-09-05T02:00:25.422455+0000","last_fullsized":"2024-09-05T02:00:25.422455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:58.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422241+0000","last_change":"2024-09-05T02:00:14.361018+0000","last_active":"2024-09-05T02:00:25.422241+0000","last_peered":"2024-09-05T02:00:25.422241+0000","last_clean":"2024-09-05T02:00:25.422241+0000","last_became_active":"2024-09-05T02:00:14.360591+0000","last_became_peered":"2024-09-05T02:00:14.360591+0000","last_unstale":"2024-09-05T02:00:25.422241+0000","last_undegraded":"2024-09-05T02:00:25.422241+0000","last_fullsized":"2024-09-05T02:00:25.422241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:49.175681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.230423+0000","last_change":"2024-09-05T02:00:12.353257+0000","last_active":"2024-09-05T02:00:58.230423+0000","last_peered":"2024-09-05T02:00:58.230423+0000","last_clean":"2024-09-05T02:00:58.230423+0000","last_became_active":"2024-09-05T02:00:12.352906+0000","last_became_peered":"2024-09-05T02:00:12.352906+0000","last_unstale":"2024-09-05T02:00:58.230423+0000","last_undegraded":"2024-09-05T02:00:58.230423+0000","last_fullsized":"2024-09-05T02:00:58.230423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:53:52.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422494+0000","last_change":"2024-09-05T02:00:21.410346+0000","last_active":"2024-09-05T02:00:25.422494+0000","last_peered":"2024-09-05T02:00:25.422494+0000","last_clean":"2024-09-05T02:00:25.422494+0000","last_became_active":"2024-09-05T02:00:21.409946+0000","last_became_peered":"2024-09-05T02:00:21.409946+0000","last_unstale":"2024-09-05T02:00:25.422494+0000","last_undegraded":"2024-09-05T02:00:25.422494+0000","last_fullsized":"2024-09-05T02:00:25.422494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:56.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415514+0000","last_change":"2024-09-05T02:00:14.360265+0000","last_active":"2024-09-05T02:00:24.415514+0000","last_peered":"2024-09-05T02:00:24.415514+0000","last_clean":"2024-09-05T02:00:24.415514+0000","last_became_active":"2024-09-05T02:00:14.359958+0000","last_became_peered":"2024-09-05T02:00:14.359958+0000","last_unstale":"2024-09-05T02:00:24.415514+0000","last_undegraded":"2024-09-05T02:00:24.415514+0000","last_fullsized":"2024-09-05T02:00:24.415514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:15:45.211207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:59.240546+0000","last_change":"2024-09-05T02:00:12.349443+0000","last_active":"2024-09-05T02:00:59.240546+0000","last_peered":"2024-09-05T02:00:59.240546+0000","last_clean":"2024-09-05T02:00:59.240546+0000","last_became_active":"2024-09-05T02:00:12.348988+0000","last_became_peered":"2024-09-05T02:00:12.348988+0000","last_unstale":"2024-09-05T02:00:59.240546+0000","last_undegraded":"2024-09-05T02:00:59.240546+0000","last_fullsized":"2024-09-05T02:00:59.240546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:14:21.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.691110+0000","last_change":"2024-09-05T02:00:21.417662+0000","last_active":"2024-09-05T02:00:24.691110+0000","last_peered":"2024-09-05T02:00:24.691110+0000","last_clean":"2024-09-05T02:00:24.691110+0000","last_became_active":"2024-09-05T02:00:21.417238+0000","last_became_peered":"2024-09-05T02:00:21.417238+0000","last_unstale":"2024-09-05T02:00:24.691110+0000","last_undegraded":"2024-09-05T02:00:24.691110+0000","last_fullsized":"2024-09-05T02:00:24.691110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:05:59.304480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421675+0000","last_change":"2024-09-05T02:00:14.355928+0000","last_active":"2024-09-05T02:00:25.421675+0000","last_peered":"2024-09-05T02:00:25.421675+0000","last_clean":"2024-09-05T02:00:25.421675+0000","last_became_active":"2024-09-05T02:00:14.355576+0000","last_became_peered":"2024-09-05T02:00:14.355576+0000","last_unstale":"2024-09-05T02:00:25.421675+0000","last_undegraded":"2024-09-05T02:00:25.421675+0000","last_fullsized":"2024-09-05T02:00:25.421675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:22:50.095434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'31","reported_seq":87,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:08.689261+0000","last_change":"2024-09-05T02:00:12.352198+0000","last_active":"2024-09-05T02:01:08.689261+0000","last_peered":"2024-09-05T02:01:08.689261+0000","last_clean":"2024-09-05T02:01:08.689261+0000","last_became_active":"2024-09-05T02:00:12.351806+0000","last_became_peered":"2024-09-05T02:00:12.351806+0000","last_unstale":"2024-09-05T02:01:08.689261+0000","last_undegraded":"2024-09-05T02:01:08.689261+0000","last_fullsized":"2024-09-05T02:01:08.689261+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.178160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52408,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":55,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421650+0000","last_change":"2024-09-05T02:00:21.416223+0000","last_active":"2024-09-05T02:00:25.421650+0000","last_peered":"2024-09-05T02:00:25.421650+0000","last_clean":"2024-09-05T02:00:25.421650+0000","last_became_active":"2024-09-05T02:00:21.415610+0000","last_became_peered":"2024-09-05T02:00:21.415610+0000","last_unstale":"2024-09-05T02:00:25.421650+0000","last_undegraded":"2024-09-05T02:00:25.421650+0000","last_fullsized":"2024-09-05T02:00:25.421650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:08:40.955438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421591+0000","last_change":"2024-09-05T02:00:14.365372+0000","last_active":"2024-09-05T02:00:25.421591+0000","last_peered":"2024-09-05T02:00:25.421591+0000","last_clean":"2024-09-05T02:00:25.421591+0000","last_became_active":"2024-09-05T02:00:14.365064+0000","last_became_peered":"2024-09-05T02:00:14.365064+0000","last_unstale":"2024-09-05T02:00:25.421591+0000","last_undegraded":"2024-09-05T02:00:25.421591+0000","last_fullsized":"2024-09-05T02:00:25.421591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:02:29.378126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:57.217191+0000","last_change":"2024-09-05T02:00:12.783487+0000","last_active":"2024-09-05T02:00:57.217191+0000","last_peered":"2024-09-05T02:00:57.217191+0000","last_clean":"2024-09-05T02:00:57.217191+0000","last_became_active":"2024-09-05T02:00:12.783087+0000","last_became_peered":"2024-09-05T02:00:12.783087+0000","last_unstale":"2024-09-05T02:00:57.217191+0000","last_undegraded":"2024-09-05T02:00:57.217191+0000","last_fullsized":"2024-09-05T02:00:57.217191+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:08.574845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423345+0000","last_change":"2024-09-05T02:00:21.419250+0000","last_active":"2024-09-05T02:00:25.423345+0000","last_peered":"2024-09-05T02:00:25.423345+0000","last_clean":"2024-09-05T02:00:25.423345+0000","last_became_active":"2024-09-05T02:00:21.418851+0000","last_became_peered":"2024-09-05T02:00:21.418851+0000","last_unstale":"2024-09-05T02:00:25.423345+0000","last_undegraded":"2024-09-05T02:00:25.423345+0000","last_fullsized":"2024-09-05T02:00:25.423345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:24:46.762006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422278+0000","last_change":"2024-09-05T02:00:14.359172+0000","last_active":"2024-09-05T02:00:25.422278+0000","last_peered":"2024-09-05T02:00:25.422278+0000","last_clean":"2024-09-05T02:00:25.422278+0000","last_became_active":"2024-09-05T02:00:14.358782+0000","last_became_peered":"2024-09-05T02:00:14.358782+0000","last_unstale":"2024-09-05T02:00:25.422278+0000","last_undegraded":"2024-09-05T02:00:25.422278+0000","last_fullsized":"2024-09-05T02:00:25.422278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:06:12.150349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.230240+0000","last_change":"2024-09-05T02:00:12.349362+0000","last_active":"2024-09-05T02:00:58.230240+0000","last_peered":"2024-09-05T02:00:58.230240+0000","last_clean":"2024-09-05T02:00:58.230240+0000","last_became_active":"2024-09-05T02:00:12.349016+0000","last_became_peered":"2024-09-05T02:00:12.349016+0000","last_unstale":"2024-09-05T02:00:58.230240+0000","last_undegraded":"2024-09-05T02:00:58.230240+0000","last_fullsized":"2024-09-05T02:00:58.230240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:37.603717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421237+0000","last_change":"2024-09-05T02:00:21.412942+0000","last_active":"2024-09-05T02:00:25.421237+0000","last_peered":"2024-09-05T02:00:25.421237+0000","last_clean":"2024-09-05T02:00:25.421237+0000","last_became_active":"2024-09-05T02:00:21.412057+0000","last_became_peered":"2024-09-05T02:00:21.412057+0000","last_unstale":"2024-09-05T02:00:25.421237+0000","last_undegraded":"2024-09-05T02:00:25.421237+0000","last_fullsized":"2024-09-05T02:00:25.421237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:59.872818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422277+0000","last_change":"2024-09-05T02:00:14.356636+0000","last_active":"2024-09-05T02:00:25.422277+0000","last_peered":"2024-09-05T02:00:25.422277+0000","last_clean":"2024-09-05T02:00:25.422277+0000","last_became_active":"2024-09-05T02:00:14.356321+0000","last_became_peered":"2024-09-05T02:00:14.356321+0000","last_unstale":"2024-09-05T02:00:25.422277+0000","last_undegraded":"2024-09-05T02:00:25.422277+0000","last_fullsized":"2024-09-05T02:00:25.422277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:02.112646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:08.254985+0000","last_change":"2024-09-05T02:00:12.353078+0000","last_active":"2024-09-05T02:01:08.254985+0000","last_peered":"2024-09-05T02:01:08.254985+0000","last_clean":"2024-09-05T02:01:08.254985+0000","last_became_active":"2024-09-05T02:00:12.352707+0000","last_became_peered":"2024-09-05T02:00:12.352707+0000","last_unstale":"2024-09-05T02:01:08.254985+0000","last_undegraded":"2024-09-05T02:01:08.254985+0000","last_fullsized":"2024-09-05T02:01:08.254985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:28:29.934304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416652+0000","last_change":"2024-09-05T02:00:21.421362+0000","last_active":"2024-09-05T02:00:24.416652+0000","last_peered":"2024-09-05T02:00:24.416652+0000","last_clean":"2024-09-05T02:00:24.416652+0000","last_became_active":"2024-09-05T02:00:21.420998+0000","last_became_peered":"2024-09-05T02:00:21.420998+0000","last_unstale":"2024-09-05T02:00:24.416652+0000","last_undegraded":"2024-09-05T02:00:24.416652+0000","last_fullsized":"2024-09-05T02:00:24.416652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:26:14.105489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421794+0000","last_change":"2024-09-05T02:00:21.416703+0000","last_active":"2024-09-05T02:00:25.421794+0000","last_peered":"2024-09-05T02:00:25.421794+0000","last_clean":"2024-09-05T02:00:25.421794+0000","last_became_active":"2024-09-05T02:00:21.415913+0000","last_became_peered":"2024-09-05T02:00:21.415913+0000","last_unstale":"2024-09-05T02:00:25.421794+0000","last_undegraded":"2024-09-05T02:00:25.421794+0000","last_fullsized":"2024-09-05T02:00:25.421794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:56.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421356+0000","last_change":"2024-09-05T02:00:14.355658+0000","last_active":"2024-09-05T02:00:25.421356+0000","last_peered":"2024-09-05T02:00:25.421356+0000","last_clean":"2024-09-05T02:00:25.421356+0000","last_became_active":"2024-09-05T02:00:14.355087+0000","last_became_peered":"2024-09-05T02:00:14.355087+0000","last_unstale":"2024-09-05T02:00:25.421356+0000","last_undegraded":"2024-09-05T02:00:25.421356+0000","last_fullsized":"2024-09-05T02:00:25.421356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:58:08.369027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:58.229348+0000","last_change":"2024-09-05T02:00:12.348171+0000","last_active":"2024-09-05T02:00:58.229348+0000","last_peered":"2024-09-05T02:00:58.229348+0000","last_clean":"2024-09-05T02:00:58.229348+0000","last_became_active":"2024-09-05T02:00:12.347760+0000","last_became_peered":"2024-09-05T02:00:12.347760+0000","last_unstale":"2024-09-05T02:00:58.229348+0000","last_undegraded":"2024-09-05T02:00:58.229348+0000","last_fullsized":"2024-09-05T02:00:58.229348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:03:43.748458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422739+0000","last_change":"2024-09-05T02:00:21.413015+0000","last_active":"2024-09-05T02:00:25.422739+0000","last_peered":"2024-09-05T02:00:25.422739+0000","last_clean":"2024-09-05T02:00:25.422739+0000","last_became_active":"2024-09-05T02:00:21.412631+0000","last_became_peered":"2024-09-05T02:00:21.412631+0000","last_unstale":"2024-09-05T02:00:25.422739+0000","last_undegraded":"2024-09-05T02:00:25.422739+0000","last_fullsized":"2024-09-05T02:00:25.422739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:35.094273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416602+0000","last_change":"2024-09-05T02:00:14.354552+0000","last_active":"2024-09-05T02:00:24.416602+0000","last_peered":"2024-09-05T02:00:24.416602+0000","last_clean":"2024-09-05T02:00:24.416602+0000","last_became_active":"2024-09-05T02:00:14.354150+0000","last_became_peered":"2024-09-05T02:00:14.354150+0000","last_unstale":"2024-09-05T02:00:24.416602+0000","last_undegraded":"2024-09-05T02:00:24.416602+0000","last_fullsized":"2024-09-05T02:00:24.416602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:16:19.836148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:56.206744+0000","last_change":"2024-09-05T02:00:12.342608+0000","last_active":"2024-09-05T02:00:56.206744+0000","last_peered":"2024-09-05T02:00:56.206744+0000","last_clean":"2024-09-05T02:00:56.206744+0000","last_became_active":"2024-09-05T02:00:12.342235+0000","last_became_peered":"2024-09-05T02:00:12.342235+0000","last_unstale":"2024-09-05T02:00:56.206744+0000","last_undegraded":"2024-09-05T02:00:56.206744+0000","last_fullsized":"2024-09-05T02:00:56.206744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:54.568963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:57.217311+0000","last_change":"2024-09-05T02:00:12.352247+0000","last_active":"2024-09-05T02:00:57.217311+0000","last_peered":"2024-09-05T02:00:57.217311+0000","last_clean":"2024-09-05T02:00:57.217311+0000","last_became_active":"2024-09-05T02:00:12.351897+0000","last_became_peered":"2024-09-05T02:00:12.351897+0000","last_unstale":"2024-09-05T02:00:57.217311+0000","last_undegraded":"2024-09-05T02:00:57.217311+0000","last_fullsized":"2024-09-05T02:00:57.217311+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:48.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690897+0000","last_change":"2024-09-05T02:00:14.366415+0000","last_active":"2024-09-05T02:00:24.690897+0000","last_peered":"2024-09-05T02:00:24.690897+0000","last_clean":"2024-09-05T02:00:24.690897+0000","last_became_active":"2024-09-05T02:00:14.366103+0000","last_became_peered":"2024-09-05T02:00:14.366103+0000","last_unstale":"2024-09-05T02:00:24.690897+0000","last_undegraded":"2024-09-05T02:00:24.690897+0000","last_fullsized":"2024-09-05T02:00:24.690897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:22:21.585867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416506+0000","last_change":"2024-09-05T02:00:21.416879+0000","last_active":"2024-09-05T02:00:24.416506+0000","last_peered":"2024-09-05T02:00:24.416506+0000","last_clean":"2024-09-05T02:00:24.416506+0000","last_became_active":"2024-09-05T02:00:21.415887+0000","last_became_peered":"2024-09-05T02:00:21.415887+0000","last_unstale":"2024-09-05T02:00:24.416506+0000","last_undegraded":"2024-09-05T02:00:24.416506+0000","last_fullsized":"2024-09-05T02:00:24.416506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:58.106529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:01:08.255511+0000","last_change":"2024-09-05T02:00:12.354055+0000","last_active":"2024-09-05T02:01:08.255511+0000","last_peered":"2024-09-05T02:01:08.255511+0000","last_clean":"2024-09-05T02:01:08.255511+0000","last_became_active":"2024-09-05T02:00:12.353754+0000","last_became_peered":"2024-09-05T02:00:12.353754+0000","last_unstale":"2024-09-05T02:01:08.255511+0000","last_undegraded":"2024-09-05T02:01:08.255511+0000","last_fullsized":"2024-09-05T02:01:08.255511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:17.671079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421856+0000","last_change":"2024-09-05T02:00:14.358646+0000","last_active":"2024-09-05T02:00:25.421856+0000","last_peered":"2024-09-05T02:00:25.421856+0000","last_clean":"2024-09-05T02:00:25.421856+0000","last_became_active":"2024-09-05T02:00:14.358337+0000","last_became_peered":"2024-09-05T02:00:14.358337+0000","last_unstale":"2024-09-05T02:00:25.421856+0000","last_undegraded":"2024-09-05T02:00:25.421856+0000","last_fullsized":"2024-09-05T02:00:25.421856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:20:39.976731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416799+0000","last_change":"2024-09-05T02:00:21.416611+0000","last_active":"2024-09-05T02:00:24.416799+0000","last_peered":"2024-09-05T02:00:24.416799+0000","last_clean":"2024-09-05T02:00:24.416799+0000","last_became_active":"2024-09-05T02:00:21.416152+0000","last_became_peered":"2024-09-05T02:00:21.416152+0000","last_unstale":"2024-09-05T02:00:24.416799+0000","last_undegraded":"2024-09-05T02:00:24.416799+0000","last_fullsized":"2024-09-05T02:00:24.416799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:20.670383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421422+0000","last_change":"2024-09-05T02:00:14.359015+0000","last_active":"2024-09-05T02:00:25.421422+0000","last_peered":"2024-09-05T02:00:25.421422+0000","last_clean":"2024-09-05T02:00:25.421422+0000","last_became_active":"2024-09-05T02:00:14.358642+0000","last_became_peered":"2024-09-05T02:00:14.358642+0000","last_unstale":"2024-09-05T02:00:25.421422+0000","last_undegraded":"2024-09-05T02:00:25.421422+0000","last_fullsized":"2024-09-05T02:00:25.421422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:18:03.723033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416457+0000","last_change":"2024-09-05T02:00:14.362391+0000","last_active":"2024-09-05T02:00:24.416457+0000","last_peered":"2024-09-05T02:00:24.416457+0000","last_clean":"2024-09-05T02:00:24.416457+0000","last_became_active":"2024-09-05T02:00:14.362007+0000","last_became_peered":"2024-09-05T02:00:14.362007+0000","last_unstale":"2024-09-05T02:00:24.416457+0000","last_undegraded":"2024-09-05T02:00:24.416457+0000","last_fullsized":"2024-09-05T02:00:24.416457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:39:02.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416177+0000","last_change":"2024-09-05T02:00:14.362769+0000","last_active":"2024-09-05T02:00:24.416177+0000","last_peered":"2024-09-05T02:00:24.416177+0000","last_clean":"2024-09-05T02:00:24.416177+0000","last_became_active":"2024-09-05T02:00:14.362310+0000","last_became_peered":"2024-09-05T02:00:14.362310+0000","last_unstale":"2024-09-05T02:00:24.416177+0000","last_undegraded":"2024-09-05T02:00:24.416177+0000","last_fullsized":"2024-09-05T02:00:24.416177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:27:11.288158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421122+0000","last_change":"2024-09-05T02:00:14.357820+0000","last_active":"2024-09-05T02:00:25.421122+0000","last_peered":"2024-09-05T02:00:25.421122+0000","last_clean":"2024-09-05T02:00:25.421122+0000","last_became_active":"2024-09-05T02:00:14.357320+0000","last_became_peered":"2024-09-05T02:00:14.357320+0000","last_unstale":"2024-09-05T02:00:25.421122+0000","last_undegraded":"2024-09-05T02:00:25.421122+0000","last_fullsized":"2024-09-05T02:00:25.421122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:39.105192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415760+0000","last_change":"2024-09-05T02:00:14.356726+0000","last_active":"2024-09-05T02:00:24.415760+0000","last_peered":"2024-09-05T02:00:24.415760+0000","last_clean":"2024-09-05T02:00:24.415760+0000","last_became_active":"2024-09-05T02:00:14.356452+0000","last_became_peered":"2024-09-05T02:00:14.356452+0000","last_unstale":"2024-09-05T02:00:24.415760+0000","last_undegraded":"2024-09-05T02:00:24.415760+0000","last_fullsized":"2024-09-05T02:00:24.415760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:34:36.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421967+0000","last_change":"2024-09-05T02:00:14.357645+0000","last_active":"2024-09-05T02:00:25.421967+0000","last_peered":"2024-09-05T02:00:25.421967+0000","last_clean":"2024-09-05T02:00:25.421967+0000","last_became_active":"2024-09-05T02:00:14.357330+0000","last_became_peered":"2024-09-05T02:00:14.357330+0000","last_unstale":"2024-09-05T02:00:25.421967+0000","last_undegraded":"2024-09-05T02:00:25.421967+0000","last_fullsized":"2024-09-05T02:00:25.421967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:42.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416761+0000","last_change":"2024-09-05T02:00:14.363413+0000","last_active":"2024-09-05T02:00:24.416761+0000","last_peered":"2024-09-05T02:00:24.416761+0000","last_clean":"2024-09-05T02:00:24.416761+0000","last_became_active":"2024-09-05T02:00:14.362821+0000","last_became_peered":"2024-09-05T02:00:14.362821+0000","last_unstale":"2024-09-05T02:00:24.416761+0000","last_undegraded":"2024-09-05T02:00:24.416761+0000","last_fullsized":"2024-09-05T02:00:24.416761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:49:39.084171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.417022+0000","last_change":"2024-09-05T02:00:14.366581+0000","last_active":"2024-09-05T02:00:24.417022+0000","last_peered":"2024-09-05T02:00:24.417022+0000","last_clean":"2024-09-05T02:00:24.417022+0000","last_became_active":"2024-09-05T02:00:14.366287+0000","last_became_peered":"2024-09-05T02:00:14.366287+0000","last_unstale":"2024-09-05T02:00:24.417022+0000","last_undegraded":"2024-09-05T02:00:24.417022+0000","last_fullsized":"2024-09-05T02:00:24.417022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:36:57.143489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.690488+0000","last_change":"2024-09-05T02:00:14.360311+0000","last_active":"2024-09-05T02:00:24.690488+0000","last_peered":"2024-09-05T02:00:24.690488+0000","last_clean":"2024-09-05T02:00:24.690488+0000","last_became_active":"2024-09-05T02:00:14.360028+0000","last_became_peered":"2024-09-05T02:00:14.360028+0000","last_unstale":"2024-09-05T02:00:24.690488+0000","last_undegraded":"2024-09-05T02:00:24.690488+0000","last_fullsized":"2024-09-05T02:00:24.690488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:35:38.804555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416656+0000","last_change":"2024-09-05T02:00:14.365853+0000","last_active":"2024-09-05T02:00:24.416656+0000","last_peered":"2024-09-05T02:00:24.416656+0000","last_clean":"2024-09-05T02:00:24.416656+0000","last_became_active":"2024-09-05T02:00:14.365549+0000","last_became_peered":"2024-09-05T02:00:14.365549+0000","last_unstale":"2024-09-05T02:00:24.416656+0000","last_undegraded":"2024-09-05T02:00:24.416656+0000","last_fullsized":"2024-09-05T02:00:24.416656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:58:45.897592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421705+0000","last_change":"2024-09-05T02:00:14.367357+0000","last_active":"2024-09-05T02:00:25.421705+0000","last_peered":"2024-09-05T02:00:25.421705+0000","last_clean":"2024-09-05T02:00:25.421705+0000","last_became_active":"2024-09-05T02:00:14.366794+0000","last_became_peered":"2024-09-05T02:00:14.366794+0000","last_unstale":"2024-09-05T02:00:25.421705+0000","last_undegraded":"2024-09-05T02:00:25.421705+0000","last_fullsized":"2024-09-05T02:00:25.421705+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:40:51.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415730+0000","last_change":"2024-09-05T02:00:14.356991+0000","last_active":"2024-09-05T02:00:24.415730+0000","last_peered":"2024-09-05T02:00:24.415730+0000","last_clean":"2024-09-05T02:00:24.415730+0000","last_became_active":"2024-09-05T02:00:14.356662+0000","last_became_peered":"2024-09-05T02:00:14.356662+0000","last_unstale":"2024-09-05T02:00:24.415730+0000","last_undegraded":"2024-09-05T02:00:24.415730+0000","last_fullsized":"2024-09-05T02:00:24.415730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:39:53.087448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422572+0000","last_change":"2024-09-05T02:00:14.357750+0000","last_active":"2024-09-05T02:00:25.422572+0000","last_peered":"2024-09-05T02:00:25.422572+0000","last_clean":"2024-09-05T02:00:25.422572+0000","last_became_active":"2024-09-05T02:00:14.357413+0000","last_became_peered":"2024-09-05T02:00:14.357413+0000","last_unstale":"2024-09-05T02:00:25.422572+0000","last_undegraded":"2024-09-05T02:00:25.422572+0000","last_fullsized":"2024-09-05T02:00:25.422572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:29:47.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416894+0000","last_change":"2024-09-05T02:00:14.364068+0000","last_active":"2024-09-05T02:00:24.416894+0000","last_peered":"2024-09-05T02:00:24.416894+0000","last_clean":"2024-09-05T02:00:24.416894+0000","last_became_active":"2024-09-05T02:00:14.363674+0000","last_became_peered":"2024-09-05T02:00:14.363674+0000","last_unstale":"2024-09-05T02:00:24.416894+0000","last_undegraded":"2024-09-05T02:00:24.416894+0000","last_fullsized":"2024-09-05T02:00:24.416894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:28:00.788668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422245+0000","last_change":"2024-09-05T02:00:14.355152+0000","last_active":"2024-09-05T02:00:25.422245+0000","last_peered":"2024-09-05T02:00:25.422245+0000","last_clean":"2024-09-05T02:00:25.422245+0000","last_became_active":"2024-09-05T02:00:14.354874+0000","last_became_peered":"2024-09-05T02:00:14.354874+0000","last_unstale":"2024-09-05T02:00:25.422245+0000","last_undegraded":"2024-09-05T02:00:25.422245+0000","last_fullsized":"2024-09-05T02:00:25.422245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:14:17.069049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422966+0000","last_change":"2024-09-05T02:00:14.360241+0000","last_active":"2024-09-05T02:00:25.422966+0000","last_peered":"2024-09-05T02:00:25.422966+0000","last_clean":"2024-09-05T02:00:25.422966+0000","last_became_active":"2024-09-05T02:00:14.359648+0000","last_became_peered":"2024-09-05T02:00:14.359648+0000","last_unstale":"2024-09-05T02:00:25.422966+0000","last_undegraded":"2024-09-05T02:00:25.422966+0000","last_fullsized":"2024-09-05T02:00:25.422966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:15.060575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421357+0000","last_change":"2024-09-05T02:00:14.366094+0000","last_active":"2024-09-05T02:00:25.421357+0000","last_peered":"2024-09-05T02:00:25.421357+0000","last_clean":"2024-09-05T02:00:25.421357+0000","last_became_active":"2024-09-05T02:00:14.365788+0000","last_became_peered":"2024-09-05T02:00:14.365788+0000","last_unstale":"2024-09-05T02:00:25.421357+0000","last_undegraded":"2024-09-05T02:00:25.421357+0000","last_fullsized":"2024-09-05T02:00:25.421357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:23:00.909487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.420957+0000","last_change":"2024-09-05T02:00:14.364651+0000","last_active":"2024-09-05T02:00:25.420957+0000","last_peered":"2024-09-05T02:00:25.420957+0000","last_clean":"2024-09-05T02:00:25.420957+0000","last_became_active":"2024-09-05T02:00:14.364365+0000","last_became_peered":"2024-09-05T02:00:14.364365+0000","last_unstale":"2024-09-05T02:00:25.420957+0000","last_undegraded":"2024-09-05T02:00:25.420957+0000","last_fullsized":"2024-09-05T02:00:25.420957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:14.978382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.420874+0000","last_change":"2024-09-05T02:00:14.364761+0000","last_active":"2024-09-05T02:00:25.420874+0000","last_peered":"2024-09-05T02:00:25.420874+0000","last_clean":"2024-09-05T02:00:25.420874+0000","last_became_active":"2024-09-05T02:00:14.364204+0000","last_became_peered":"2024-09-05T02:00:14.364204+0000","last_unstale":"2024-09-05T02:00:25.420874+0000","last_undegraded":"2024-09-05T02:00:25.420874+0000","last_fullsized":"2024-09-05T02:00:25.420874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:16:51.076304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.415636+0000","last_change":"2024-09-05T02:00:14.357632+0000","last_active":"2024-09-05T02:00:24.415636+0000","last_peered":"2024-09-05T02:00:24.415636+0000","last_clean":"2024-09-05T02:00:24.415636+0000","last_became_active":"2024-09-05T02:00:14.357348+0000","last_became_peered":"2024-09-05T02:00:14.357348+0000","last_unstale":"2024-09-05T02:00:24.415636+0000","last_undegraded":"2024-09-05T02:00:24.415636+0000","last_fullsized":"2024-09-05T02:00:24.415636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:11.163292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423340+0000","last_change":"2024-09-05T02:00:14.361484+0000","last_active":"2024-09-05T02:00:25.423340+0000","last_peered":"2024-09-05T02:00:25.423340+0000","last_clean":"2024-09-05T02:00:25.423340+0000","last_became_active":"2024-09-05T02:00:14.361156+0000","last_became_peered":"2024-09-05T02:00:14.361156+0000","last_unstale":"2024-09-05T02:00:25.423340+0000","last_undegraded":"2024-09-05T02:00:25.423340+0000","last_fullsized":"2024-09-05T02:00:25.423340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:40.124733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422605+0000","last_change":"2024-09-05T02:00:14.358325+0000","last_active":"2024-09-05T02:00:25.422605+0000","last_peered":"2024-09-05T02:00:25.422605+0000","last_clean":"2024-09-05T02:00:25.422605+0000","last_became_active":"2024-09-05T02:00:14.357993+0000","last_became_peered":"2024-09-05T02:00:14.357993+0000","last_unstale":"2024-09-05T02:00:25.422605+0000","last_undegraded":"2024-09-05T02:00:25.422605+0000","last_fullsized":"2024-09-05T02:00:25.422605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:18.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.614040+0000","last_change":"2024-09-05T02:00:14.365291+0000","last_active":"2024-09-05T02:00:24.614040+0000","last_peered":"2024-09-05T02:00:24.614040+0000","last_clean":"2024-09-05T02:00:24.614040+0000","last_became_active":"2024-09-05T02:00:14.364937+0000","last_became_peered":"2024-09-05T02:00:14.364937+0000","last_unstale":"2024-09-05T02:00:24.614040+0000","last_undegraded":"2024-09-05T02:00:24.614040+0000","last_fullsized":"2024-09-05T02:00:24.614040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:42:57.712651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422643+0000","last_change":"2024-09-05T02:00:14.367603+0000","last_active":"2024-09-05T02:00:25.422643+0000","last_peered":"2024-09-05T02:00:25.422643+0000","last_clean":"2024-09-05T02:00:25.422643+0000","last_became_active":"2024-09-05T02:00:14.367267+0000","last_became_peered":"2024-09-05T02:00:14.367267+0000","last_unstale":"2024-09-05T02:00:25.422643+0000","last_undegraded":"2024-09-05T02:00:25.422643+0000","last_fullsized":"2024-09-05T02:00:25.422643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:51:34.595898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.416590+0000","last_change":"2024-09-05T02:00:14.369499+0000","last_active":"2024-09-05T02:00:24.416590+0000","last_peered":"2024-09-05T02:00:24.416590+0000","last_clean":"2024-09-05T02:00:24.416590+0000","last_became_active":"2024-09-05T02:00:14.369169+0000","last_became_peered":"2024-09-05T02:00:14.369169+0000","last_unstale":"2024-09-05T02:00:24.416590+0000","last_undegraded":"2024-09-05T02:00:24.416590+0000","last_fullsized":"2024-09-05T02:00:24.416590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:17.161204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422019+0000","last_change":"2024-09-05T02:00:14.360320+0000","last_active":"2024-09-05T02:00:25.422019+0000","last_peered":"2024-09-05T02:00:25.422019+0000","last_clean":"2024-09-05T02:00:25.422019+0000","last_became_active":"2024-09-05T02:00:14.359991+0000","last_became_peered":"2024-09-05T02:00:14.359991+0000","last_unstale":"2024-09-05T02:00:25.422019+0000","last_undegraded":"2024-09-05T02:00:25.422019+0000","last_fullsized":"2024-09-05T02:00:25.422019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:03.721863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:24.613563+0000","last_change":"2024-09-05T02:00:14.364660+0000","last_active":"2024-09-05T02:00:24.613563+0000","last_peered":"2024-09-05T02:00:24.613563+0000","last_clean":"2024-09-05T02:00:24.613563+0000","last_became_active":"2024-09-05T02:00:14.364313+0000","last_became_peered":"2024-09-05T02:00:14.364313+0000","last_unstale":"2024-09-05T02:00:24.613563+0000","last_undegraded":"2024-09-05T02:00:24.613563+0000","last_fullsized":"2024-09-05T02:00:24.613563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:47.496063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421354+0000","last_change":"2024-09-05T02:00:14.359853+0000","last_active":"2024-09-05T02:00:25.421354+0000","last_peered":"2024-09-05T02:00:25.421354+0000","last_clean":"2024-09-05T02:00:25.421354+0000","last_became_active":"2024-09-05T02:00:14.359543+0000","last_became_peered":"2024-09-05T02:00:14.359543+0000","last_unstale":"2024-09-05T02:00:25.421354+0000","last_undegraded":"2024-09-05T02:00:25.421354+0000","last_fullsized":"2024-09-05T02:00:25.421354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:30:29.415492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.423281+0000","last_change":"2024-09-05T02:00:14.363836+0000","last_active":"2024-09-05T02:00:25.423281+0000","last_peered":"2024-09-05T02:00:25.423281+0000","last_clean":"2024-09-05T02:00:25.423281+0000","last_became_active":"2024-09-05T02:00:14.363481+0000","last_became_peered":"2024-09-05T02:00:14.363481+0000","last_unstale":"2024-09-05T02:00:25.423281+0000","last_undegraded":"2024-09-05T02:00:25.423281+0000","last_fullsized":"2024-09-05T02:00:25.423281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:46.443519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421556+0000","last_change":"2024-09-05T02:00:14.367191+0000","last_active":"2024-09-05T02:00:25.421556+0000","last_peered":"2024-09-05T02:00:25.421556+0000","last_clean":"2024-09-05T02:00:25.421556+0000","last_became_active":"2024-09-05T02:00:14.366831+0000","last_became_peered":"2024-09-05T02:00:14.366831+0000","last_unstale":"2024-09-05T02:00:25.421556+0000","last_undegraded":"2024-09-05T02:00:25.421556+0000","last_fullsized":"2024-09-05T02:00:25.421556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:55.690642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.421894+0000","last_change":"2024-09-05T02:00:14.355774+0000","last_active":"2024-09-05T02:00:25.421894+0000","last_peered":"2024-09-05T02:00:25.421894+0000","last_clean":"2024-09-05T02:00:25.421894+0000","last_became_active":"2024-09-05T02:00:14.355293+0000","last_became_peered":"2024-09-05T02:00:14.355293+0000","last_unstale":"2024-09-05T02:00:25.421894+0000","last_undegraded":"2024-09-05T02:00:25.421894+0000","last_fullsized":"2024-09-05T02:00:25.421894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:07:00.547820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T02:00:25.422874+0000","last_change":"2024-09-05T02:00:14.359978+0000","last_active":"2024-09-05T02:00:25.422874+0000","last_peered":"2024-09-05T02:00:25.422874+0000","last_clean":"2024-09-05T02:00:25.422874+0000","last_became_active":"2024-09-05T02:00:14.359209+0000","last_became_peered":"2024-09-05T02:00:14.359209+0000","last_unstale":"2024-09-05T02:00:25.422874+0000","last_undegraded":"2024-09-05T02:00:25.422874+0000","last_fullsized":"2024-09-05T02:00:25.422874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:27.848906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59616,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":143,"num_read_kb":127,"num_write":238,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":178848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":478731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1117562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972387,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":482364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201320,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":478646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430252,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1075145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":473772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36908,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708244,"statfs":{"total":95995035648,"available":95957241856,"internally_reserved":0,"allocated":1601536,"data_stored":1070727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":477846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":473480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444101,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":528216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":527210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":475621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T02:01:15.241 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T02:01:15.242 INFO:tasks.cephadm:Setting up client nodes... 2024-09-05T02:01:15.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-05T02:01:15.522 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:16.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.993+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/2164539205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb881026d0 msgr2=0x7fdb88108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:16.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.993+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/2164539205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb881026d0 0x7fdb88108f60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009ca0 tx=0x7fdb8002a8c0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.993+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/2164539205 shutdown_connections 2024-09-05T02:01:16.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.993+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/2164539205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb881026d0 0x7fdb88108f60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.993+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/2164539205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb88101d10 0x7fdb88102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.993+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/2164539205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb88100b10 0x7fdb88100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.993+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/2164539205 >> 172.21.15.103:0/2164539205 conn(0x7fdb880fc2a0 msgr2=0x7fdb880fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/2164539205 shutdown_connections 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/2164539205 wait complete. 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 Processor -- start 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 -- start start 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb88100b10 0x7fdb8806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88101d10 0x7fdb8819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881026d0 0x7fdb881a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810c690 con 0x7fdb88101d10 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810c510 con 0x7fdb88100b10 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.994+0000 7fdb8e8b4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810c810 con 0x7fdb881026d0 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88101d10 0x7fdb8819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88101d10 0x7fdb8819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41742/0 (socket says 172.21.15.103:41742) 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb877fe640 1 -- 172.21.15.103:0/1412493229 learned_addr learned my addr 172.21.15.103:0/1412493229 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb87fff640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb88100b10 0x7fdb8806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb8ce51640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881026d0 0x7fdb881a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb877fe640 1 -- 172.21.15.103:0/1412493229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881026d0 msgr2=0x7fdb881a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb877fe640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881026d0 0x7fdb881a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb877fe640 1 -- 172.21.15.103:0/1412493229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb88100b10 msgr2=0x7fdb8806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb877fe640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb88100b10 0x7fdb8806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb877fe640 1 -- 172.21.15.103:0/1412493229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb881a1c00 con 0x7fdb88101d10 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb877fe640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88101d10 0x7fdb8819eec0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00d770 tx=0x7fdb7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb857fa640 1 -- 172.21.15.103:0/1412493229 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c017070 con 0x7fdb88101d10 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb857fa640 1 -- 172.21.15.103:0/1412493229 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb7c0054d0 con 0x7fdb88101d10 2024-09-05T02:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb881a1e90 con 0x7fdb88101d10 2024-09-05T02:01:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb857fa640 1 -- 172.21.15.103:0/1412493229 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c004cf0 con 0x7fdb88101d10 2024-09-05T02:01:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.995+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb881a2340 con 0x7fdb88101d10 2024-09-05T02:01:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.996+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb88107ec0 con 0x7fdb88101d10 2024-09-05T02:01:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.997+0000 7fdb857fa640 1 -- 172.21.15.103:0/1412493229 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdb7c004e90 con 0x7fdb88101d10 2024-09-05T02:01:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.998+0000 7fdb857fa640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb700a2cc0 0x7fdb700a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.998+0000 7fdb87fff640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb700a2cc0 0x7fdb700a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.998+0000 7fdb857fa640 1 -- 172.21.15.103:0/1412493229 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fdb7c0ced30 con 0x7fdb88101d10 2024-09-05T02:01:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:15.999+0000 7fdb87fff640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb700a2cc0 0x7fdb700a5180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb88101b70 tx=0x7fdb78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.003+0000 7fdb857fa640 1 -- 172.21.15.103:0/1412493229 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb7c0d3070 con 0x7fdb88101d10 2024-09-05T02:01:16.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.232+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fdb88062df0 con 0x7fdb88101d10 2024-09-05T02:01:16.233 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:16 smithi103 ceph-mon[30902]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:16.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:16 smithi103 ceph-mon[30902]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-09-05T02:01:16.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.234+0000 7fdb857fa640 1 -- 172.21.15.103:0/1412493229 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fdb7c09b2a0 con 0x7fdb88101d10 2024-09-05T02:01:16.235 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-09-05T02:01:16.235 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDsENlmYTTwDRAA9IWB8zZXYoiZYz3kVPdeNQ== 2024-09-05T02:01:16.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.236+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb700a2cc0 msgr2=0x7fdb700a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:16.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.236+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb700a2cc0 0x7fdb700a5180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb88101b70 tx=0x7fdb78005f90 comp rx=0 tx=0).stop 2024-09-05T02:01:16.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.236+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88101d10 msgr2=0x7fdb8819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:16.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.236+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88101d10 0x7fdb8819eec0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00d770 tx=0x7fdb7c00dc30 comp rx=0 tx=0).stop 2024-09-05T02:01:16.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.237+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 shutdown_connections 2024-09-05T02:01:16.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.237+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb700a2cc0 0x7fdb700a5180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.237+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881026d0 0x7fdb881a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.237+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88101d10 0x7fdb8819eec0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.237+0000 7fdb8e8b4640 1 --2- 172.21.15.103:0/1412493229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb88100b10 0x7fdb8806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.237+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 >> 172.21.15.103:0/1412493229 conn(0x7fdb880fc2a0 msgr2=0x7fdb88105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.237+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 shutdown_connections 2024-09-05T02:01:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:16.237+0000 7fdb8e8b4640 1 -- 172.21.15.103:0/1412493229 wait complete. 2024-09-05T02:01:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:16 smithi167 ceph-mon[35122]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:16 smithi167 ceph-mon[35122]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-09-05T02:01:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:16 smithi139 ceph-mon[34262]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:16.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:16 smithi139 ceph-mon[34262]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-09-05T02:01:16.905 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T02:01:16.905 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-05T02:01:16.905 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-05T02:01:16.942 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-05T02:01:16.942 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-05T02:01:16.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph mgr dump --format=json 2024-09-05T02:01:17.227 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:17.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:17 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1412493229' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T02:01:17.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:17 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1412493229' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T02:01:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:17 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1412493229' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T02:01:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:17 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1412493229' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T02:01:17.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:17 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1412493229' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T02:01:17.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:17 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1412493229' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T02:01:17.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.709+0000 7f9528353640 1 -- 172.21.15.103:0/2404394878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102570 msgr2=0x7f95201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:17.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.709+0000 7f9528353640 1 --2- 172.21.15.103:0/2404394878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102570 0x7f95201029b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9510009900 tx=0x7f951002a480 comp rx=0 tx=0).stop 2024-09-05T02:01:17.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.710+0000 7f9528353640 1 -- 172.21.15.103:0/2404394878 shutdown_connections 2024-09-05T02:01:17.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.710+0000 7f9528353640 1 --2- 172.21.15.103:0/2404394878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9520102ef0 0x7f952010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.710+0000 7f9528353640 1 --2- 172.21.15.103:0/2404394878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102570 0x7f95201029b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.710+0000 7f9528353640 1 --2- 172.21.15.103:0/2404394878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9520100e20 0x7f9520101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.710+0000 7f9528353640 1 -- 172.21.15.103:0/2404394878 >> 172.21.15.103:0/2404394878 conn(0x7f95200fc7d0 msgr2=0x7f95200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:17.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.710+0000 7f9528353640 1 -- 172.21.15.103:0/2404394878 shutdown_connections 2024-09-05T02:01:17.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.710+0000 7f9528353640 1 -- 172.21.15.103:0/2404394878 wait complete. 2024-09-05T02:01:17.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.710+0000 7f9528353640 1 Processor -- start 2024-09-05T02:01:17.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.710+0000 7f9528353640 1 -- start start 2024-09-05T02:01:17.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f9528353640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9520100e20 0x7f95201983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:17.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f9528353640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9520102570 0x7f9520198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:17.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f9528353640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102ef0 0x7f952019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:17.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f9528353640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f952006ae70 con 0x7f9520102ef0 2024-09-05T02:01:17.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f9528353640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f952006acf0 con 0x7f9520100e20 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f9528353640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9520112110 con 0x7f9520102570 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95268f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102ef0 0x7f952019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95268f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102ef0 0x7f952019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41762/0 (socket says 172.21.15.103:41762) 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95268f0640 1 -- 172.21.15.103:0/2595151618 learned_addr learned my addr 172.21.15.103:0/2595151618 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95260ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9520100e20 0x7f95201983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95258ee640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9520102570 0x7f9520198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95268f0640 1 -- 172.21.15.103:0/2595151618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9520102570 msgr2=0x7f9520198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95268f0640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9520102570 0x7f9520198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95268f0640 1 -- 172.21.15.103:0/2595151618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9520100e20 msgr2=0x7f95201983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:17.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95268f0640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9520100e20 0x7f95201983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95268f0640 1 -- 172.21.15.103:0/2595151618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952019d370 con 0x7f9520102ef0 2024-09-05T02:01:17.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95268f0640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102ef0 0x7f952019cc90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f951800cad0 tx=0x7f951800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:17.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.711+0000 7f95177fe640 1 -- 172.21.15.103:0/2595151618 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951800ad60 con 0x7f9520102ef0 2024-09-05T02:01:17.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.712+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952019d660 con 0x7f9520102ef0 2024-09-05T02:01:17.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.712+0000 7f95177fe640 1 -- 172.21.15.103:0/2595151618 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9518005020 con 0x7f9520102ef0 2024-09-05T02:01:17.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.712+0000 7f95177fe640 1 -- 172.21.15.103:0/2595151618 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951800f6b0 con 0x7f9520102ef0 2024-09-05T02:01:17.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.712+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95201a4f40 con 0x7f9520102ef0 2024-09-05T02:01:17.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.713+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9520101200 con 0x7f9520102ef0 2024-09-05T02:01:17.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.714+0000 7f95177fe640 1 -- 172.21.15.103:0/2595151618 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f95180051d0 con 0x7f9520102ef0 2024-09-05T02:01:17.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.714+0000 7f95177fe640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f950c0a2c10 0x7f950c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:17.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.714+0000 7f95260ef640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f950c0a2c10 0x7f950c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:17.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.714+0000 7f95177fe640 1 -- 172.21.15.103:0/2595151618 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f95180cf480 con 0x7f9520102ef0 2024-09-05T02:01:17.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.715+0000 7f95260ef640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f950c0a2c10 0x7f950c0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f951c00abe0 tx=0x7f951c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:17.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.719+0000 7f95177fe640 1 -- 172.21.15.103:0/2595151618 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9518014030 con 0x7f9520102ef0 2024-09-05T02:01:17.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.929+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9520109090 con 0x7f9520102ef0 2024-09-05T02:01:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.934+0000 7f95177fe640 1 -- 172.21.15.103:0/2595151618 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263208 (secure 0 0 0) 0x7f951809c040 con 0x7f9520102ef0 2024-09-05T02:01:17.940 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f950c0a2c10 msgr2=0x7f950c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f950c0a2c10 0x7f950c0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f951c00abe0 tx=0x7f951c00a2d0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102ef0 msgr2=0x7f952019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102ef0 0x7f952019cc90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f951800cad0 tx=0x7f951800ced0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 shutdown_connections 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9520102ef0 0x7f952019cc90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f950c0a2c10 0x7f950c0a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9520102570 0x7f9520198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 --2- 172.21.15.103:0/2595151618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9520100e20 0x7f95201983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:17.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 >> 172.21.15.103:0/2595151618 conn(0x7f95200fc7d0 msgr2=0x7f95200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:17.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 shutdown_connections 2024-09-05T02:01:17.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:17.936+0000 7f9528353640 1 -- 172.21.15.103:0/2595151618 wait complete. 2024-09-05T02:01:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:18 smithi167 ceph-mon[35122]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-05T02:01:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:18 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2595151618' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T02:01:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:18 smithi139 ceph-mon[34262]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-05T02:01:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:18 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2595151618' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T02:01:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:18 smithi103 ceph-mon[30902]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-05T02:01:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:18 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2595151618' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T02:01:18.724 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4267002703},{"type":"v1","addr":"172.21.15.103:6801","nonce":4267002703}]},"active_addr":"172.21.15.103:6801/4267002703","active_change":"2024-09-05T01:53:25.001929+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.103:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":3372898792}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2954782588}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2062660682}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":389701921}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":434457160}]}]} 2024-09-05T02:01:18.730 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-05T02:01:18.730 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-05T02:01:18.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:01:18.977 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:19.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.488+0000 7fd1255ec640 1 -- 172.21.15.103:0/1641601639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120108160 msgr2=0x7fd12010eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.488+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1641601639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120108160 0x7fd12010eb00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd118009cd0 tx=0x7fd11802a8e0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.488+0000 7fd1255ec640 1 -- 172.21.15.103:0/1641601639 shutdown_connections 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.488+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1641601639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120108160 0x7fd12010eb00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.488+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1641601639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd120101290 0x7fd120107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.488+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1641601639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1201008e0 0x7fd120100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.488+0000 7fd1255ec640 1 -- 172.21.15.103:0/1641601639 >> 172.21.15.103:0/1641601639 conn(0x7fd120077210 msgr2=0x7fd120077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.488+0000 7fd1255ec640 1 -- 172.21.15.103:0/1641601639 shutdown_connections 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.488+0000 7fd1255ec640 1 -- 172.21.15.103:0/1641601639 wait complete. 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.489+0000 7fd1255ec640 1 Processor -- start 2024-09-05T02:01:19.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.489+0000 7fd1255ec640 1 -- start start 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.489+0000 7fd1255ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1201008e0 0x7fd120101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.489+0000 7fd1255ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120101290 0x7fd120101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.489+0000 7fd1255ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd120108160 0x7fd1201022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.489+0000 7fd1255ec640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd120111ea0 con 0x7fd120101290 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.489+0000 7fd1255ec640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd120111d20 con 0x7fd120108160 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.489+0000 7fd1255ec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd120112020 con 0x7fd1201008e0 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.489+0000 7fd11e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120101290 0x7fd120101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120101290 0x7fd120101da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35558/0 (socket says 172.21.15.103:35558) 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11e7fc640 1 -- 172.21.15.103:0/1233649934 learned_addr learned my addr 172.21.15.103:0/1233649934 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11effd640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1201008e0 0x7fd120101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11f7fe640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd120108160 0x7fd1201022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11e7fc640 1 -- 172.21.15.103:0/1233649934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1201008e0 msgr2=0x7fd120101860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11e7fc640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1201008e0 0x7fd120101860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11e7fc640 1 -- 172.21.15.103:0/1233649934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd120108160 msgr2=0x7fd1201022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11e7fc640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd120108160 0x7fd1201022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11e7fc640 1 -- 172.21.15.103:0/1233649934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1201a67d0 con 0x7fd120101290 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11f7fe640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd120108160 0x7fd1201022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd11e7fc640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120101290 0x7fd120101da0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd11400e9f0 tx=0x7fd11400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd107fff640 1 -- 172.21.15.103:0/1233649934 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11400ccc0 con 0x7fd120101290 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1201a6ac0 con 0x7fd120101290 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd107fff640 1 -- 172.21.15.103:0/1233649934 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1140054d0 con 0x7fd120101290 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd107fff640 1 -- 172.21.15.103:0/1233649934 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114010430 con 0x7fd120101290 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.490+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1201a7000 con 0x7fd120101290 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.491+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd120100cc0 con 0x7fd120101290 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.493+0000 7fd107fff640 1 -- 172.21.15.103:0/1233649934 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd114004fb0 con 0x7fd120101290 2024-09-05T02:01:19.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.494+0000 7fd107fff640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd1080a2c90 0x7fd1080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:19.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.494+0000 7fd11effd640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd1080a2c90 0x7fd1080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:19.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.494+0000 7fd107fff640 1 -- 172.21.15.103:0/1233649934 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fd1140cf720 con 0x7fd120101290 2024-09-05T02:01:19.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.495+0000 7fd11effd640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd1080a2c90 0x7fd1080a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd11000abe0 tx=0x7fd11000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:19.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.498+0000 7fd107fff640 1 -- 172.21.15.103:0/1233649934 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd11409c2e0 con 0x7fd120101290 2024-09-05T02:01:19.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.673+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd120109be0 con 0x7fd120101290 2024-09-05T02:01:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.674+0000 7fd107fff640 1 -- 172.21.15.103:0/1233649934 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7fd11409bbe0 con 0x7fd120101290 2024-09-05T02:01:19.676 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:19.676 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:00:24.085875+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:01:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd1080a2c90 msgr2=0x7fd1080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd1080a2c90 0x7fd1080a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd11000abe0 tx=0x7fd11000a2d0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120101290 msgr2=0x7fd120101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120101290 0x7fd120101da0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd11400e9f0 tx=0x7fd11400edf0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 shutdown_connections 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd120108160 0x7fd1201022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd120101290 0x7fd120101da0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd1080a2c90 0x7fd1080a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 --2- 172.21.15.103:0/1233649934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1201008e0 0x7fd120101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 >> 172.21.15.103:0/1233649934 conn(0x7fd120077210 msgr2=0x7fd1200ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 shutdown_connections 2024-09-05T02:01:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:19.677+0000 7fd1255ec640 1 -- 172.21.15.103:0/1233649934 wait complete. 2024-09-05T02:01:20.348 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:20 smithi103 ceph-mon[30902]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-05T02:01:20.348 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:20 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1233649934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:01:20.375 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-05T02:01:20.376 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:01:20.626 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:20 smithi167 ceph-mon[35122]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-05T02:01:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:20 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1233649934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:01:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:20 smithi139 ceph-mon[34262]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-05T02:01:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:20 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1233649934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:01:21.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.114+0000 7f99abc8e640 1 -- 172.21.15.103:0/1001420617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4101420 msgr2=0x7f99a410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.114+0000 7f99abc8e640 1 --2- 172.21.15.103:0/1001420617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4101420 0x7f99a410d950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f999c009cd0 tx=0x7f999c02a8e0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 -- 172.21.15.103:0/1001420617 shutdown_connections 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 --2- 172.21.15.103:0/1001420617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4101420 0x7f99a410d950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 --2- 172.21.15.103:0/1001420617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4100a80 0x7f99a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 --2- 172.21.15.103:0/1001420617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4106a80 0x7f99a4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 -- 172.21.15.103:0/1001420617 >> 172.21.15.103:0/1001420617 conn(0x7f99a40fc7a0 msgr2=0x7f99a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 -- 172.21.15.103:0/1001420617 shutdown_connections 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 -- 172.21.15.103:0/1001420617 wait complete. 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 Processor -- start 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 -- start start 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4100a80 0x7f99a41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4101420 0x7f99a4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:21.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4106a80 0x7f99a419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.115+0000 7f99abc8e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4110d30 con 0x7f99a4100a80 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99abc8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4110bb0 con 0x7f99a4101420 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99abc8e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4110eb0 con 0x7f99a4106a80 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9a2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4100a80 0x7f99a41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9a2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4100a80 0x7f99a41982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35574/0 (socket says 172.21.15.103:35574) 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9a2a640 1 -- 172.21.15.103:0/965988236 learned_addr learned my addr 172.21.15.103:0/965988236 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99aa22b640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4106a80 0x7f99a419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9a2a640 1 -- 172.21.15.103:0/965988236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4106a80 msgr2=0x7f99a419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9a2a640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4106a80 0x7f99a419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9a2a640 1 -- 172.21.15.103:0/965988236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4101420 msgr2=0x7f99a4198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:21.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9229640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4101420 0x7f99a4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9a2a640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4101420 0x7f99a4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9a2a640 1 -- 172.21.15.103:0/965988236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a419d270 con 0x7f99a4100a80 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99a9a2a640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4100a80 0x7f99a41982c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f99a000c890 tx=0x7f99a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f999affd640 1 -- 172.21.15.103:0/965988236 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0004220 con 0x7f99a4100a80 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a419d560 con 0x7f99a4100a80 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f999affd640 1 -- 172.21.15.103:0/965988236 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99a00043c0 con 0x7f99a4100a80 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f999affd640 1 -- 172.21.15.103:0/965988236 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0004a00 con 0x7f99a4100a80 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.116+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a41a4e40 con 0x7f99a4100a80 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.117+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a4100ee0 con 0x7f99a4100a80 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.119+0000 7f999affd640 1 -- 172.21.15.103:0/965988236 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f99a0004ba0 con 0x7f99a4100a80 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.119+0000 7f999affd640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c10 0x7f99900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.119+0000 7f999affd640 1 -- 172.21.15.103:0/965988236 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f99a00cf570 con 0x7f99a4100a80 2024-09-05T02:01:21.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.119+0000 7f99a9229640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c10 0x7f99900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:21.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.120+0000 7f99a9229640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c10 0x7f99900a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99a406b7e0 tx=0x7f9994005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:21.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.122+0000 7f999affd640 1 -- 172.21.15.103:0/965988236 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99a0014030 con 0x7f99a4100a80 2024-09-05T02:01:21.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.294+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99a41a51e0 con 0x7f99a4100a80 2024-09-05T02:01:21.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.296+0000 7f999affd640 1 -- 172.21.15.103:0/965988236 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f99a009c130 con 0x7f99a4100a80 2024-09-05T02:01:21.298 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:21.298 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:00:24.085875+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:01:21.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.298+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c10 msgr2=0x7f99900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:21.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.298+0000 7f99abc8e640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c10 0x7f99900a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99a406b7e0 tx=0x7f9994005cc0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.298+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4100a80 msgr2=0x7f99a41982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:21.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.298+0000 7f99abc8e640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4100a80 0x7f99a41982c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f99a000c890 tx=0x7f99a000cd50 comp rx=0 tx=0).stop 2024-09-05T02:01:21.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.299+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 shutdown_connections 2024-09-05T02:01:21.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.299+0000 7f99abc8e640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f99900a2c10 0x7f99900a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.299+0000 7f99abc8e640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4106a80 0x7f99a419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.299+0000 7f99abc8e640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4101420 0x7f99a4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.299+0000 7f99abc8e640 1 --2- 172.21.15.103:0/965988236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99a4100a80 0x7f99a41982c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:21.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.299+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 >> 172.21.15.103:0/965988236 conn(0x7f99a40fc7a0 msgr2=0x7f99a41048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:21.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.299+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 shutdown_connections 2024-09-05T02:01:21.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:21.299+0000 7f99abc8e640 1 -- 172.21.15.103:0/965988236 wait complete. 2024-09-05T02:01:22.022 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-05T02:01:22.023 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-05T02:01:22.023 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-05T02:01:22.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-05T02:01:22.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-05T02:01:22.025 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-05T02:01:22.025 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-05T02:01:22.025 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-05T02:01:22.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-05T02:01:22.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-05T02:01:22.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-05T02:01:22.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-05T02:01:22.244 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:22 smithi103 ceph-mon[30902]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-05T02:01:22.298 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:22 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/965988236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:01:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:22 smithi167 ceph-mon[35122]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-05T02:01:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:22 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/965988236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:01:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:22 smithi139 ceph-mon[34262]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-05T02:01:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:22 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/965988236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:01:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.309+0000 7fe0a3309640 1 -- 172.21.15.103:0/3204450958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe09c1082a0 msgr2=0x7fe09c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.309+0000 7fe0a3309640 1 --2- 172.21.15.103:0/3204450958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe09c1082a0 0x7fe09c10ec40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe094009cd0 tx=0x7fe09402a8e0 comp rx=0 tx=0).stop 2024-09-05T02:01:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.309+0000 7fe0a3309640 1 -- 172.21.15.103:0/3204450958 shutdown_connections 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.309+0000 7fe0a3309640 1 --2- 172.21.15.103:0/3204450958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe09c1082a0 0x7fe09c10ec40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.309+0000 7fe0a3309640 1 --2- 172.21.15.103:0/3204450958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe09c1036e0 0x7fe09c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.309+0000 7fe0a3309640 1 --2- 172.21.15.103:0/3204450958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe09c102d30 0x7fe09c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.309+0000 7fe0a3309640 1 -- 172.21.15.103:0/3204450958 >> 172.21.15.103:0/3204450958 conn(0x7fe09c0fc7d0 msgr2=0x7fe09c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.309+0000 7fe0a3309640 1 -- 172.21.15.103:0/3204450958 shutdown_connections 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.309+0000 7fe0a3309640 1 -- 172.21.15.103:0/3204450958 wait complete. 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a3309640 1 Processor -- start 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a3309640 1 -- start start 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a3309640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe09c102d30 0x7fe09c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a3309640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe09c1036e0 0x7fe09c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a3309640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe09c1082a0 0x7fe09c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a3309640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c111f50 con 0x7fe09c1082a0 2024-09-05T02:01:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a3309640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c111dd0 con 0x7fe09c102d30 2024-09-05T02:01:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a3309640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1120d0 con 0x7fe09c1036e0 2024-09-05T02:01:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a18a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe09c1082a0 0x7fe09c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a18a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe09c1082a0 0x7fe09c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35594/0 (socket says 172.21.15.103:35594) 2024-09-05T02:01:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a18a6640 1 -- 172.21.15.103:0/116610941 learned_addr learned my addr 172.21.15.103:0/116610941 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.310+0000 7fe0a10a5640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe09c102d30 0x7fe09c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe0a08a4640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe09c1036e0 0x7fe09c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe0a18a6640 1 -- 172.21.15.103:0/116610941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe09c1036e0 msgr2=0x7fe09c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe0a18a6640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe09c1036e0 0x7fe09c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe0a18a6640 1 -- 172.21.15.103:0/116610941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe09c102d30 msgr2=0x7fe09c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe0a18a6640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe09c102d30 0x7fe09c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe0a18a6640 1 -- 172.21.15.103:0/116610941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c19f590 con 0x7fe09c1082a0 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe0a18a6640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe09c1082a0 0x7fe09c19eeb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe094005cc0 tx=0x7fe094003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe08e7fc640 1 -- 172.21.15.103:0/116610941 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094004450 con 0x7fe09c1082a0 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c19f820 con 0x7fe09c1082a0 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe08e7fc640 1 -- 172.21.15.103:0/116610941 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe094031c90 con 0x7fe09c1082a0 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe08e7fc640 1 -- 172.21.15.103:0/116610941 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09403b400 con 0x7fe09c1082a0 2024-09-05T02:01:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.311+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c1a7160 con 0x7fe09c1082a0 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.312+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe09c062980 con 0x7fe09c1082a0 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.313+0000 7fe08e7fc640 1 -- 172.21.15.103:0/116610941 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe094004680 con 0x7fe09c1082a0 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.314+0000 7fe08e7fc640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0880a2c10 0x7fe0880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.314+0000 7fe0a10a5640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0880a2c10 0x7fe0880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.314+0000 7fe08e7fc640 1 -- 172.21.15.103:0/116610941 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fe09402e030 con 0x7fe09c1082a0 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.314+0000 7fe08e7fc640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] conn(0x7fe0880ad940 0x7fe0880afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.314+0000 7fe08e7fc640 1 -- 172.21.15.103:0/116610941 --> [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0880b0450 con 0x7fe0880ad940 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.314+0000 7fe08e7fc640 1 -- 172.21.15.103:0/116610941 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe0940f0c50 con 0x7fe09c1082a0 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.314+0000 7fe0a10a5640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0880a2c10 0x7fe0880a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe090005fd0 tx=0x7fe09000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.314+0000 7fe0a08a4640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] conn(0x7fe0880ad940 0x7fe0880afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.315+0000 7fe0a08a4640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] conn(0x7fe0880ad940 0x7fe0880afd80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.316+0000 7fe08e7fc640 1 -- 172.21.15.103:0/116610941 <== osd.5 v2:172.21.15.167:6808/4050841663 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0880b0450 con 0x7fe0880ad940 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.329+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 --> [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe09c062e00 con 0x7fe0880ad940 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.329+0000 7fe08e7fc640 1 -- 172.21.15.103:0/116610941 <== osd.5 v2:172.21.15.167:6808/4050841663 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe09c062e00 con 0x7fe0880ad940 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 >> [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] conn(0x7fe0880ad940 msgr2=0x7fe0880afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] conn(0x7fe0880ad940 0x7fe0880afd80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0880a2c10 msgr2=0x7fe0880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0880a2c10 0x7fe0880a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe090005fd0 tx=0x7fe09000b390 comp rx=0 tx=0).stop 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe09c1082a0 msgr2=0x7fe09c19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 --2- 172.21.15.103:0/116610941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe09c1082a0 0x7fe09c19eeb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe094005cc0 tx=0x7fe094003c20 comp rx=0 tx=0).stop 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a18a6640 1 -- 172.21.15.103:0/116610941 reap_dead start 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 shutdown_connections 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 >> 172.21.15.103:0/116610941 conn(0x7fe09c0fc7d0 msgr2=0x7fe09c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 shutdown_connections 2024-09-05T02:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:23.330+0000 7fe0a3309640 1 -- 172.21.15.103:0/116610941 wait complete. 2024-09-05T02:01:23.979 INFO:teuthology.orchestra.run.smithi103.stdout:154618822713 2024-09-05T02:01:23.980 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-05T02:01:24.000 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.032 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.033 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.033 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.039 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.039 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:24.429 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:24 smithi103 ceph-mon[30902]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-05T02:01:24.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:24 smithi167 ceph-mon[35122]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-05T02:01:24.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:24 smithi139 ceph-mon[34262]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-05T02:01:25.358 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-05T02:01:25.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-09-05T02:01:25.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-05T02:01:25.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-05T02:01:25.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-05T02:01:25.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-05T02:01:25.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-05T02:01:25.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-05T02:01:25.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-09-05T02:01:25.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:25 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-05T02:01:25.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.358+0000 7fa6d4999640 1 -- 172.21.15.103:0/3835011483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6d0102570 msgr2=0x7fa6d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.358+0000 7fa6d4999640 1 --2- 172.21.15.103:0/3835011483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6d0102570 0x7fa6d01029b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009900 tx=0x7fa6c002a480 comp rx=0 tx=0).stop 2024-09-05T02:01:25.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.358+0000 7fa6d4999640 1 -- 172.21.15.103:0/3835011483 shutdown_connections 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.358+0000 7fa6d4999640 1 --2- 172.21.15.103:0/3835011483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0102ef0 0x7fa6d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.358+0000 7fa6d4999640 1 --2- 172.21.15.103:0/3835011483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6d0102570 0x7fa6d01029b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.358+0000 7fa6d4999640 1 --2- 172.21.15.103:0/3835011483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0100e20 0x7fa6d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.358+0000 7fa6d4999640 1 -- 172.21.15.103:0/3835011483 >> 172.21.15.103:0/3835011483 conn(0x7fa6d00fc7d0 msgr2=0x7fa6d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.358+0000 7fa6d4999640 1 -- 172.21.15.103:0/3835011483 shutdown_connections 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.358+0000 7fa6d4999640 1 -- 172.21.15.103:0/3835011483 wait complete. 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6d4999640 1 Processor -- start 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6d4999640 1 -- start start 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6d4999640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0100e20 0x7fa6d0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6d4999640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6d0102570 0x7fa6d01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6d4999640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0102ef0 0x7fa6d019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6d4999640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0110e30 con 0x7fa6d0102570 2024-09-05T02:01:25.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6d4999640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0110cb0 con 0x7fa6d0102ef0 2024-09-05T02:01:25.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6d4999640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0110fb0 con 0x7fa6d0100e20 2024-09-05T02:01:25.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6cdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6d0102570 0x7fa6d01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6cdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6d0102570 0x7fa6d01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35626/0 (socket says 172.21.15.103:35626) 2024-09-05T02:01:25.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6cdd9b640 1 -- 172.21.15.103:0/454076634 learned_addr learned my addr 172.21.15.103:0/454076634 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:25.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6ced9d640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0102ef0 0x7fa6d019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6ce59c640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0100e20 0x7fa6d0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6cdd9b640 1 -- 172.21.15.103:0/454076634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0100e20 msgr2=0x7fa6d0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6cdd9b640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0100e20 0x7fa6d0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6cdd9b640 1 -- 172.21.15.103:0/454076634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0102ef0 msgr2=0x7fa6d019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6cdd9b640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0102ef0 0x7fa6d019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.359+0000 7fa6cdd9b640 1 -- 172.21.15.103:0/454076634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d019d420 con 0x7fa6d0102570 2024-09-05T02:01:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.360+0000 7fa6ce59c640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0100e20 0x7fa6d0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.360+0000 7fa6cdd9b640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6d0102570 0x7fa6d01989b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0006010 tx=0x7fa6c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.360+0000 7fa6c77fe640 1 -- 172.21.15.103:0/454076634 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c003f030 con 0x7fa6d0102570 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.360+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d019d6b0 con 0x7fa6d0102570 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.360+0000 7fa6c77fe640 1 -- 172.21.15.103:0/454076634 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6c0003cb0 con 0x7fa6d0102570 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.360+0000 7fa6c77fe640 1 -- 172.21.15.103:0/454076634 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0035070 con 0x7fa6d0102570 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.360+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d01a4ff0 con 0x7fa6d0102570 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.361+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa6d0062980 con 0x7fa6d0102570 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.362+0000 7fa6c77fe640 1 -- 172.21.15.103:0/454076634 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa6c0042030 con 0x7fa6d0102570 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.363+0000 7fa6c77fe640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa6bc0a2c10 0x7fa6bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.363+0000 7fa6ce59c640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa6bc0a2c10 0x7fa6bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.363+0000 7fa6c77fe640 1 -- 172.21.15.103:0/454076634 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa6c00f0100 con 0x7fa6d0102570 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.363+0000 7fa6c77fe640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] conn(0x7fa6bc0adde0 0x7fa6bc0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.363+0000 7fa6c77fe640 1 -- 172.21.15.103:0/454076634 --> [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6bc0b0950 con 0x7fa6bc0adde0 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.363+0000 7fa6c77fe640 1 -- 172.21.15.103:0/454076634 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fa6c00f0550 con 0x7fa6d0102570 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.363+0000 7fa6ced9d640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] conn(0x7fa6bc0adde0 0x7fa6bc0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.363+0000 7fa6ce59c640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa6bc0a2c10 0x7fa6bc0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005fd0 tx=0x7fa6b8005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.363+0000 7fa6ced9d640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] conn(0x7fa6bc0adde0 0x7fa6bc0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.364+0000 7fa6c77fe640 1 -- 172.21.15.103:0/454076634 <== osd.9 v2:172.21.15.103:6826/3249722651 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa6bc0b0950 con 0x7fa6bc0adde0 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 --> [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa6d0062d80 con 0x7fa6bc0adde0 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6c77fe640 1 -- 172.21.15.103:0/454076634 <== osd.9 v2:172.21.15.103:6826/3249722651 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa6d0062d80 con 0x7fa6bc0adde0 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] conn(0x7fa6bc0adde0 msgr2=0x7fa6bc0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] conn(0x7fa6bc0adde0 0x7fa6bc0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa6bc0a2c10 msgr2=0x7fa6bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa6bc0a2c10 0x7fa6bc0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005fd0 tx=0x7fa6b8005ea0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6d0102570 msgr2=0x7fa6d01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 --2- 172.21.15.103:0/454076634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6d0102570 0x7fa6d01989b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0006010 tx=0x7fa6c00042f0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6ced9d640 1 -- 172.21.15.103:0/454076634 reap_dead start 2024-09-05T02:01:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 shutdown_connections 2024-09-05T02:01:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 >> 172.21.15.103:0/454076634 conn(0x7fa6d00fc7d0 msgr2=0x7fa6d006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 shutdown_connections 2024-09-05T02:01:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.378+0000 7fa6d4999640 1 -- 172.21.15.103:0/454076634 wait complete. 2024-09-05T02:01:25.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-05T02:01:25.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-09-05T02:01:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-05T02:01:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-05T02:01:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-05T02:01:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-05T02:01:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-05T02:01:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-05T02:01:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-09-05T02:01:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:25 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-05T02:01:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-05T02:01:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-09-05T02:01:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-05T02:01:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-05T02:01:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-05T02:01:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-05T02:01:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-05T02:01:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-05T02:01:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-09-05T02:01:25.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:25 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-05T02:01:25.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.246+0000 7f750c0ad640 1 -- 172.21.15.103:0/2000947666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7504101df0 msgr2=0x7f75041021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.246+0000 7f750c0ad640 1 --2- 172.21.15.103:0/2000947666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7504101df0 0x7f75041021d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009900 tx=0x7f74fc02a410 comp rx=0 tx=0).stop 2024-09-05T02:01:25.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 -- 172.21.15.103:0/2000947666 shutdown_connections 2024-09-05T02:01:25.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 --2- 172.21.15.103:0/2000947666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7504106f20 0x7f750410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 --2- 172.21.15.103:0/2000947666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75041027a0 0x7f75041067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 --2- 172.21.15.103:0/2000947666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7504101df0 0x7f75041021d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 -- 172.21.15.103:0/2000947666 >> 172.21.15.103:0/2000947666 conn(0x7f75040fd8b0 msgr2=0x7f75040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 -- 172.21.15.103:0/2000947666 shutdown_connections 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 -- 172.21.15.103:0/2000947666 wait complete. 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 Processor -- start 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 -- start start 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7504101df0 0x7f7504198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.247+0000 7f750c0ad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75041027a0 0x7f7504198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f750c0ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7504106f20 0x7f750419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f750c0ad640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7504111df0 con 0x7f75041027a0 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f750c0ad640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7504111c70 con 0x7f7504101df0 2024-09-05T02:01:25.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f750c0ad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7504111f70 con 0x7f7504106f20 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509648640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75041027a0 0x7f7504198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509648640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75041027a0 0x7f7504198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35608/0 (socket says 172.21.15.103:35608) 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509648640 1 -- 172.21.15.103:0/611721933 learned_addr learned my addr 172.21.15.103:0/611721933 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509e49640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7504101df0 0x7f7504198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f750a64a640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7504106f20 0x7f750419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509648640 1 -- 172.21.15.103:0/611721933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7504106f20 msgr2=0x7f750419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509648640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7504106f20 0x7f750419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509648640 1 -- 172.21.15.103:0/611721933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7504101df0 msgr2=0x7f7504198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509648640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7504101df0 0x7f7504198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509648640 1 -- 172.21.15.103:0/611721933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750419d1e0 con 0x7f75041027a0 2024-09-05T02:01:25.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509e49640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7504101df0 0x7f7504198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f750a64a640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7504106f20 0x7f750419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.248+0000 7f7509648640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75041027a0 0x7f7504198770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f74f800cce0 tx=0x7f74f800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.251+0000 7f7502ffd640 1 -- 172.21.15.103:0/611721933 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f800f650 con 0x7f75041027a0 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.252+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750419d4d0 con 0x7f75041027a0 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.252+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75041a4db0 con 0x7f75041027a0 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.252+0000 7f7502ffd640 1 -- 172.21.15.103:0/611721933 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74f800f7f0 con 0x7f75041027a0 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.252+0000 7f7502ffd640 1 -- 172.21.15.103:0/611721933 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8005280 con 0x7f75041027a0 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.253+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7504062980 con 0x7f75041027a0 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.254+0000 7f7502ffd640 1 -- 172.21.15.103:0/611721933 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f74f800fc30 con 0x7f75041027a0 2024-09-05T02:01:25.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.254+0000 7f7502ffd640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f74ec0a2c10 0x7f74ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.254+0000 7f7509e49640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f74ec0a2c10 0x7f74ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.254+0000 7f7502ffd640 1 -- 172.21.15.103:0/611721933 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f74f80cfc80 con 0x7f75041027a0 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.255+0000 7f7502ffd640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] conn(0x7f74ec0ad940 0x7f74ec0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.255+0000 7f7502ffd640 1 -- 172.21.15.103:0/611721933 --> [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74ec0b0450 con 0x7f74ec0ad940 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.255+0000 7f7502ffd640 1 -- 172.21.15.103:0/611721933 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f74f80d00d0 con 0x7f75041027a0 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.255+0000 7f750a64a640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] conn(0x7f74ec0ad940 0x7f74ec0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.255+0000 7f7509e49640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f74ec0a2c10 0x7f74ec0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f74fc031440 tx=0x7f74fc0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.255+0000 7f750a64a640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] conn(0x7f74ec0ad940 0x7f74ec0afd80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.257+0000 7f7502ffd640 1 -- 172.21.15.103:0/611721933 <== osd.1 v2:172.21.15.139:6800/491231014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74ec0b0450 con 0x7f74ec0ad940 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.276+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 --> [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7504062d80 con 0x7f74ec0ad940 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.276+0000 7f7502ffd640 1 -- 172.21.15.103:0/611721933 <== osd.1 v2:172.21.15.139:6800/491231014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7504062d80 con 0x7f74ec0ad940 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.276+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 >> [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] conn(0x7f74ec0ad940 msgr2=0x7f74ec0afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750c0ad640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] conn(0x7f74ec0ad940 0x7f74ec0afd80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f74ec0a2c10 msgr2=0x7f74ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750c0ad640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f74ec0a2c10 0x7f74ec0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f74fc031440 tx=0x7f74fc0095f0 comp rx=0 tx=0).stop 2024-09-05T02:01:25.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75041027a0 msgr2=0x7f7504198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:25.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750c0ad640 1 --2- 172.21.15.103:0/611721933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75041027a0 0x7f7504198770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f74f800cce0 tx=0x7f74f800a620 comp rx=0 tx=0).stop 2024-09-05T02:01:25.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750a64a640 1 -- 172.21.15.103:0/611721933 reap_dead start 2024-09-05T02:01:25.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 shutdown_connections 2024-09-05T02:01:25.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 >> 172.21.15.103:0/611721933 conn(0x7f75040fd8b0 msgr2=0x7f75040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:25.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 shutdown_connections 2024-09-05T02:01:25.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:25.277+0000 7f750c0ad640 1 -- 172.21.15.103:0/611721933 wait complete. 2024-09-05T02:01:26.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.095+0000 7f774ab1b640 1 -- 172.21.15.103:0/2993957900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7744101450 msgr2=0x7f7744105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:26.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.095+0000 7f774ab1b640 1 --2- 172.21.15.103:0/2993957900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7744101450 0x7f7744105920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7738009900 tx=0x7f773802a470 comp rx=0 tx=0).stop 2024-09-05T02:01:26.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 -- 172.21.15.103:0/2993957900 shutdown_connections 2024-09-05T02:01:26.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 --2- 172.21.15.103:0/2993957900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7744105f40 0x7f77441063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:26.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 --2- 172.21.15.103:0/2993957900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7744101450 0x7f7744105920 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 --2- 172.21.15.103:0/2993957900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7744100b30 0x7f7744100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 -- 172.21.15.103:0/2993957900 >> 172.21.15.103:0/2993957900 conn(0x7f77440fc7a0 msgr2=0x7f77440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 -- 172.21.15.103:0/2993957900 shutdown_connections 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 -- 172.21.15.103:0/2993957900 wait complete. 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 Processor -- start 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 -- start start 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7744100b30 0x7f77441a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7744101450 0x7f77441a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7744105f40 0x7f774419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7744118410 con 0x7f7744105f40 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7744118290 con 0x7f7744101450 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.096+0000 7f774ab1b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7744118590 con 0x7f7744100b30 2024-09-05T02:01:26.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77490b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7744105f40 0x7f774419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77490b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7744105f40 0x7f774419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35676/0 (socket says 172.21.15.103:35676) 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77490b8640 1 -- 172.21.15.103:0/2864281995 learned_addr learned my addr 172.21.15.103:0/2864281995 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f7743fff640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7744101450 0x7f77441a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77488b7640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7744100b30 0x7f77441a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77490b8640 1 -- 172.21.15.103:0/2864281995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7744100b30 msgr2=0x7f77441a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77490b8640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7744100b30 0x7f77441a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77490b8640 1 -- 172.21.15.103:0/2864281995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7744101450 msgr2=0x7f77441a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77490b8640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7744101450 0x7f77441a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77490b8640 1 -- 172.21.15.103:0/2864281995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774419ef90 con 0x7f7744105f40 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f77490b8640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7744105f40 0x7f774419e860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f773c009fd0 tx=0x7f773c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f7741ffb640 1 -- 172.21.15.103:0/2864281995 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c017070 con 0x7f7744105f40 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f7741ffb640 1 -- 172.21.15.103:0/2864281995 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f773c003e20 con 0x7f7744105f40 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f7741ffb640 1 -- 172.21.15.103:0/2864281995 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c010460 con 0x7f7744105f40 2024-09-05T02:01:26.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.097+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774419f280 con 0x7f7744105f40 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.098+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77441ab570 con 0x7f7744105f40 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.099+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77441052b0 con 0x7f7744105f40 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.100+0000 7f7741ffb640 1 -- 172.21.15.103:0/2864281995 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f773c003970 con 0x7f7744105f40 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.100+0000 7f7741ffb640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f772c0a2c90 0x7f772c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.100+0000 7f77488b7640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f772c0a2c90 0x7f772c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.100+0000 7f7741ffb640 1 -- 172.21.15.103:0/2864281995 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f773c0cf590 con 0x7f7744105f40 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.101+0000 7f7741ffb640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] conn(0x7f772c0ade50 0x7f772c0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.101+0000 7f7741ffb640 1 -- 172.21.15.103:0/2864281995 --> [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f773c01de50 con 0x7f772c0ade50 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.101+0000 7f7741ffb640 1 -- 172.21.15.103:0/2864281995 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f773c0cf9e0 con 0x7f7744105f40 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.101+0000 7f7743fff640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] conn(0x7f772c0ade50 0x7f772c0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.101+0000 7f77488b7640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f772c0a2c90 0x7f772c0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f774419f9a0 tx=0x7f773400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.101+0000 7f7743fff640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] conn(0x7f772c0ade50 0x7f772c0b02f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:26.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.102+0000 7f7741ffb640 1 -- 172.21.15.103:0/2864281995 <== osd.3 v2:172.21.15.103:6810/845945713 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f773c01de50 con 0x7f772c0ade50 2024-09-05T02:01:26.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.115+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 --> [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f774406b070 con 0x7f772c0ade50 2024-09-05T02:01:26.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.115+0000 7f7741ffb640 1 -- 172.21.15.103:0/2864281995 <== osd.3 v2:172.21.15.103:6810/845945713 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f774406b070 con 0x7f772c0ade50 2024-09-05T02:01:26.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] conn(0x7f772c0ade50 msgr2=0x7f772c0b02f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:26.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] conn(0x7f772c0ade50 0x7f772c0b02f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:26.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f772c0a2c90 msgr2=0x7f772c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:26.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f772c0a2c90 0x7f772c0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f774419f9a0 tx=0x7f773400b390 comp rx=0 tx=0).stop 2024-09-05T02:01:26.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7744105f40 msgr2=0x7f774419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:26.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 --2- 172.21.15.103:0/2864281995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7744105f40 0x7f774419e860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f773c009fd0 tx=0x7f773c00b260 comp rx=0 tx=0).stop 2024-09-05T02:01:26.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f77490b8640 1 -- 172.21.15.103:0/2864281995 reap_dead start 2024-09-05T02:01:26.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 shutdown_connections 2024-09-05T02:01:26.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 >> 172.21.15.103:0/2864281995 conn(0x7f77440fc7a0 msgr2=0x7f77441038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:26.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 shutdown_connections 2024-09-05T02:01:26.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.116+0000 7f774ab1b640 1 -- 172.21.15.103:0/2864281995 wait complete. 2024-09-05T02:01:26.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-05T02:01:26.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:26 smithi103 ceph-mon[30902]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T02:01:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-05T02:01:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:26 smithi167 ceph-mon[35122]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T02:01:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-05T02:01:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-05T02:01:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-09-05T02:01:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-05T02:01:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-05T02:01:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-05T02:01:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-05T02:01:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-05T02:01:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-05T02:01:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-09-05T02:01:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-05T02:01:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:26 smithi139 ceph-mon[34262]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T02:01:26.999 INFO:teuthology.orchestra.run.smithi103.stdout:103079215172 2024-09-05T02:01:26.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-05T02:01:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:27 smithi167 ceph-mon[35122]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T02:01:27.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:27 smithi139 ceph-mon[34262]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T02:01:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:27 smithi103 ceph-mon[30902]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T02:01:27.694 INFO:teuthology.orchestra.run.smithi103.stdout:266287972391 2024-09-05T02:01:27.695 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-05T02:01:28.117 INFO:teuthology.orchestra.run.smithi103.stdout:55834574926 2024-09-05T02:01:28.118 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-05T02:01:28.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.281+0000 7f46b63fa640 1 -- 172.21.15.103:0/3570944428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b0101420 msgr2=0x7f46b010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.281+0000 7f46b63fa640 1 --2- 172.21.15.103:0/3570944428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b0101420 0x7f46b010d950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009cd0 tx=0x7f46a802a8e0 comp rx=0 tx=0).stop 2024-09-05T02:01:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.281+0000 7f46b63fa640 1 -- 172.21.15.103:0/3570944428 shutdown_connections 2024-09-05T02:01:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.281+0000 7f46b63fa640 1 --2- 172.21.15.103:0/3570944428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b0101420 0x7f46b010d950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.281+0000 7f46b63fa640 1 --2- 172.21.15.103:0/3570944428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0100a80 0x7f46b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.281+0000 7f46b63fa640 1 --2- 172.21.15.103:0/3570944428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b0106a80 0x7f46b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.281+0000 7f46b63fa640 1 -- 172.21.15.103:0/3570944428 >> 172.21.15.103:0/3570944428 conn(0x7f46b00fc7a0 msgr2=0x7f46b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.281+0000 7f46b63fa640 1 -- 172.21.15.103:0/3570944428 shutdown_connections 2024-09-05T02:01:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.281+0000 7f46b63fa640 1 -- 172.21.15.103:0/3570944428 wait complete. 2024-09-05T02:01:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.282+0000 7f46b63fa640 1 Processor -- start 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.282+0000 7f46b63fa640 1 -- start start 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b63fa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0100a80 0x7f46b0197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b63fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b0101420 0x7f46b0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b63fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b0106a80 0x7f46b019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b63fa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0110d70 con 0x7f46b0106a80 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b63fa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0110bf0 con 0x7f46b0101420 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b63fa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0110ef0 con 0x7f46b0100a80 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b4997640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b0106a80 0x7f46b019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b4997640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b0106a80 0x7f46b019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35686/0 (socket says 172.21.15.103:35686) 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b4997640 1 -- 172.21.15.103:0/1557825023 learned_addr learned my addr 172.21.15.103:0/1557825023 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46affff640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0100a80 0x7f46b0197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46af7fe640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b0101420 0x7f46b0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:28.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b4997640 1 -- 172.21.15.103:0/1557825023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0100a80 msgr2=0x7f46b0197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b4997640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0100a80 0x7f46b0197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b4997640 1 -- 172.21.15.103:0/1557825023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b0101420 msgr2=0x7f46b0197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b4997640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b0101420 0x7f46b0197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.283+0000 7f46b4997640 1 -- 172.21.15.103:0/1557825023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b019c2c0 con 0x7f46b0106a80 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.284+0000 7f46b4997640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b0106a80 0x7f46b019bbe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f46a8005cc0 tx=0x7f46a8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.284+0000 7f46ad7fa640 1 -- 172.21.15.103:0/1557825023 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a804a070 con 0x7f46b0106a80 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.284+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b019c550 con 0x7f46b0106a80 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.284+0000 7f46ad7fa640 1 -- 172.21.15.103:0/1557825023 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46a8031c90 con 0x7f46b0106a80 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.284+0000 7f46ad7fa640 1 -- 172.21.15.103:0/1557825023 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a80307e0 con 0x7f46b0106a80 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.284+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b006a790 con 0x7f46b0106a80 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.285+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46b006ad40 con 0x7f46b0106a80 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.286+0000 7f46ad7fa640 1 -- 172.21.15.103:0/1557825023 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f46a8030980 con 0x7f46b0106a80 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.287+0000 7f46ad7fa640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46980a2c10 0x7f46980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:28.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.287+0000 7f46affff640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46980a2c10 0x7f46980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:28.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.287+0000 7f46ad7fa640 1 -- 172.21.15.103:0/1557825023 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f46a802e020 con 0x7f46b0106a80 2024-09-05T02:01:28.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.287+0000 7f46ad7fa640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] conn(0x7f46980adde0 0x7f46980b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:28.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.287+0000 7f46ad7fa640 1 -- 172.21.15.103:0/1557825023 --> [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46a80020b0 con 0x7f46980adde0 2024-09-05T02:01:28.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.287+0000 7f46ad7fa640 1 -- 172.21.15.103:0/1557825023 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f46a80f23a0 con 0x7f46b0106a80 2024-09-05T02:01:28.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.287+0000 7f46af7fe640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] conn(0x7f46980adde0 0x7f46980b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:28.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.287+0000 7f46affff640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46980a2c10 0x7f46980a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f46a0005fd0 tx=0x7f46a000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:28.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.288+0000 7f46af7fe640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] conn(0x7f46980adde0 0x7f46980b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:28.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.289+0000 7f46ad7fa640 1 -- 172.21.15.103:0/1557825023 <== osd.8 v2:172.21.15.167:6816/639772211 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46a80020b0 con 0x7f46980adde0 2024-09-05T02:01:28.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.303+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 --> [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46b006b1c0 con 0x7f46980adde0 2024-09-05T02:01:28.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46ad7fa640 1 -- 172.21.15.103:0/1557825023 <== osd.8 v2:172.21.15.167:6816/639772211 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f46b006b1c0 con 0x7f46980adde0 2024-09-05T02:01:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 >> [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] conn(0x7f46980adde0 msgr2=0x7f46980b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] conn(0x7f46980adde0 0x7f46980b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46980a2c10 msgr2=0x7f46980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46980a2c10 0x7f46980a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f46a0005fd0 tx=0x7f46a000b390 comp rx=0 tx=0).stop 2024-09-05T02:01:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b0106a80 msgr2=0x7f46b019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 --2- 172.21.15.103:0/1557825023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b0106a80 0x7f46b019bbe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f46a8005cc0 tx=0x7f46a8003ba0 comp rx=0 tx=0).stop 2024-09-05T02:01:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b4997640 1 -- 172.21.15.103:0/1557825023 reap_dead start 2024-09-05T02:01:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 shutdown_connections 2024-09-05T02:01:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 >> 172.21.15.103:0/1557825023 conn(0x7f46b00fc7a0 msgr2=0x7f46b00fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:28.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 shutdown_connections 2024-09-05T02:01:28.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:28.304+0000 7f46b63fa640 1 -- 172.21.15.103:0/1557825023 wait complete. 2024-09-05T02:01:28.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:28 smithi103 ceph-mon[30902]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T02:01:28.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:28 smithi103 ceph-mon[30902]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-05T02:01:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:28 smithi167 ceph-mon[35122]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T02:01:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:28 smithi167 ceph-mon[35122]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-05T02:01:28.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:28 smithi139 ceph-mon[34262]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T02:01:28.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:28 smithi139 ceph-mon[34262]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.001+0000 7fa7838cb640 1 -- 172.21.15.103:0/3243331988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa77c100b30 msgr2=0x7fa77c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.001+0000 7fa7838cb640 1 --2- 172.21.15.103:0/3243331988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa77c100b30 0x7fa77c100f10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa778009900 tx=0x7fa77802a470 comp rx=0 tx=0).stop 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.001+0000 7fa7838cb640 1 -- 172.21.15.103:0/3243331988 shutdown_connections 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.001+0000 7fa7838cb640 1 --2- 172.21.15.103:0/3243331988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c105f40 0x7fa77c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.001+0000 7fa7838cb640 1 --2- 172.21.15.103:0/3243331988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c101450 0x7fa77c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.001+0000 7fa7838cb640 1 --2- 172.21.15.103:0/3243331988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa77c100b30 0x7fa77c100f10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.001+0000 7fa7838cb640 1 -- 172.21.15.103:0/3243331988 >> 172.21.15.103:0/3243331988 conn(0x7fa77c0fc7a0 msgr2=0x7fa77c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.001+0000 7fa7838cb640 1 -- 172.21.15.103:0/3243331988 shutdown_connections 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.001+0000 7fa7838cb640 1 -- 172.21.15.103:0/3243331988 wait complete. 2024-09-05T02:01:29.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa7838cb640 1 Processor -- start 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa7838cb640 1 -- start start 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa7838cb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c100b30 0x7fa77c1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa7838cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa77c101450 0x7fa77c1a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa7838cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c105f40 0x7fa77c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa7838cb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c118400 con 0x7fa77c101450 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa7838cb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c118280 con 0x7fa77c105f40 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa7838cb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c118580 con 0x7fa77c100b30 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa780e66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa77c101450 0x7fa77c1a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa780e66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa77c101450 0x7fa77c1a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35654/0 (socket says 172.21.15.103:35654) 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa780e66640 1 -- 172.21.15.103:0/3149368743 learned_addr learned my addr 172.21.15.103:0/3149368743 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.002+0000 7fa781667640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c100b30 0x7fa77c1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa781e68640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c105f40 0x7fa77c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:29.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa780e66640 1 -- 172.21.15.103:0/3149368743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c100b30 msgr2=0x7fa77c1a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:29.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa780e66640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c100b30 0x7fa77c1a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:29.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa780e66640 1 -- 172.21.15.103:0/3149368743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c105f40 msgr2=0x7fa77c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:29.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa780e66640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c105f40 0x7fa77c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:29.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa780e66640 1 -- 172.21.15.103:0/3149368743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c19f020 con 0x7fa77c101450 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa781667640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77c100b30 0x7fa77c1a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa781e68640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c105f40 0x7fa77c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa780e66640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa77c101450 0x7fa77c1a4bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d740 tx=0x7fa76c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa7727fc640 1 -- 172.21.15.103:0/3149368743 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c017070 con 0x7fa77c101450 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa7727fc640 1 -- 172.21.15.103:0/3149368743 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa76c00de50 con 0x7fa77c101450 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c19f310 con 0x7fa77c101450 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa7727fc640 1 -- 172.21.15.103:0/3149368743 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00f620 con 0x7fa77c101450 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.003+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c1ab680 con 0x7fa77c101450 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.004+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa77c1100a0 con 0x7fa77c101450 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.005+0000 7fa7727fc640 1 -- 172.21.15.103:0/3149368743 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa76c009c30 con 0x7fa77c101450 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.006+0000 7fa7727fc640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa7680a2c10 0x7fa7680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.006+0000 7fa781667640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa7680a2c10 0x7fa7680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.006+0000 7fa7727fc640 1 -- 172.21.15.103:0/3149368743 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa76c0cf550 con 0x7fa77c101450 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.006+0000 7fa7727fc640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] conn(0x7fa7680adde0 0x7fa7680b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.006+0000 7fa7727fc640 1 -- 172.21.15.103:0/3149368743 --> [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa76c010a00 con 0x7fa7680adde0 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.006+0000 7fa7727fc640 1 -- 172.21.15.103:0/3149368743 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fa76c0cf9a0 con 0x7fa77c101450 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.006+0000 7fa781667640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa7680a2c10 0x7fa7680a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa778031440 tx=0x7fa778005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.007+0000 7fa781e68640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] conn(0x7fa7680adde0 0x7fa7680b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.007+0000 7fa781e68640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] conn(0x7fa7680adde0 0x7fa7680b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.008+0000 7fa7727fc640 1 -- 172.21.15.103:0/3149368743 <== osd.10 v2:172.21.15.139:6824/3256384109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa76c010a00 con 0x7fa7680adde0 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.022+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 --> [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa77c06b070 con 0x7fa7680adde0 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7727fc640 1 -- 172.21.15.103:0/3149368743 <== osd.10 v2:172.21.15.139:6824/3256384109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa77c06b070 con 0x7fa7680adde0 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 >> [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] conn(0x7fa7680adde0 msgr2=0x7fa7680b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:29.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] conn(0x7fa7680adde0 0x7fa7680b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:29.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa7680a2c10 msgr2=0x7fa7680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:29.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa7680a2c10 0x7fa7680a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa778031440 tx=0x7fa778005a20 comp rx=0 tx=0).stop 2024-09-05T02:01:29.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa77c101450 msgr2=0x7fa77c1a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:29.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 --2- 172.21.15.103:0/3149368743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa77c101450 0x7fa77c1a4bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d740 tx=0x7fa76c00dc00 comp rx=0 tx=0).stop 2024-09-05T02:01:29.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa781e68640 1 -- 172.21.15.103:0/3149368743 reap_dead start 2024-09-05T02:01:29.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 shutdown_connections 2024-09-05T02:01:29.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 >> 172.21.15.103:0/3149368743 conn(0x7fa77c0fc7a0 msgr2=0x7fa77c10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:29.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 shutdown_connections 2024-09-05T02:01:29.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:26.023+0000 7fa7838cb640 1 -- 172.21.15.103:0/3149368743 wait complete. 2024-09-05T02:01:29.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:29 smithi167 ceph-mon[35122]: pgmap v320: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-05T02:01:29.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:29 smithi139 ceph-mon[34262]: pgmap v320: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-05T02:01:29.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:29 smithi103 ceph-mon[30902]: pgmap v320: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-05T02:01:29.730 INFO:teuthology.orchestra.run.smithi103.stdout:287762808865 2024-09-05T02:01:29.731 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-05T02:01:30.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.015+0000 7fae39bd9640 1 -- 172.21.15.103:0/1949026237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae34102790 msgr2=0x7fae341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.015+0000 7fae39bd9640 1 --2- 172.21.15.103:0/1949026237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae34102790 0x7fae341067e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fae28009900 tx=0x7fae2802a410 comp rx=0 tx=0).stop 2024-09-05T02:01:30.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.015+0000 7fae39bd9640 1 -- 172.21.15.103:0/1949026237 shutdown_connections 2024-09-05T02:01:30.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.015+0000 7fae39bd9640 1 --2- 172.21.15.103:0/1949026237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae34106f10 0x7fae3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.015+0000 7fae39bd9640 1 --2- 172.21.15.103:0/1949026237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae34102790 0x7fae341067e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.015+0000 7fae39bd9640 1 --2- 172.21.15.103:0/1949026237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae34101de0 0x7fae341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.015+0000 7fae39bd9640 1 -- 172.21.15.103:0/1949026237 >> 172.21.15.103:0/1949026237 conn(0x7fae340fd8c0 msgr2=0x7fae340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.015+0000 7fae39bd9640 1 -- 172.21.15.103:0/1949026237 shutdown_connections 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.015+0000 7fae39bd9640 1 -- 172.21.15.103:0/1949026237 wait complete. 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.016+0000 7fae39bd9640 1 Processor -- start 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.016+0000 7fae39bd9640 1 -- start start 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae39bd9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae34101de0 0x7fae341983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae39bd9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae34102790 0x7fae34198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae39bd9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae34106f10 0x7fae3419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae39bd9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae34111e20 con 0x7fae34106f10 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae39bd9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae34111ca0 con 0x7fae34102790 2024-09-05T02:01:30.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae39bd9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae34111fa0 con 0x7fae34101de0 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae34106f10 0x7fae3419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae34106f10 0x7fae3419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47390/0 (socket says 172.21.15.103:47390) 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae33fff640 1 -- 172.21.15.103:0/1043077780 learned_addr learned my addr 172.21.15.103:0/1043077780 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae32ffd640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae34102790 0x7fae34198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae337fe640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae34101de0 0x7fae341983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae33fff640 1 -- 172.21.15.103:0/1043077780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae34101de0 msgr2=0x7fae341983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae33fff640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae34101de0 0x7fae341983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae33fff640 1 -- 172.21.15.103:0/1043077780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae34102790 msgr2=0x7fae34198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae33fff640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae34102790 0x7fae34198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae33fff640 1 -- 172.21.15.103:0/1043077780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3419d370 con 0x7fae34106f10 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae337fe640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae34101de0 0x7fae341983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:30.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae32ffd640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae34102790 0x7fae34198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae33fff640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae34106f10 0x7fae3419cc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00eb10 tx=0x7fae2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae30ff9640 1 -- 172.21.15.103:0/1043077780 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c004b20 con 0x7fae34106f10 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3419d660 con 0x7fae34106f10 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae30ff9640 1 -- 172.21.15.103:0/1043077780 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae2c004cc0 con 0x7fae34106f10 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae30ff9640 1 -- 172.21.15.103:0/1043077780 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c010690 con 0x7fae34106f10 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.017+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae341a4f40 con 0x7fae34106f10 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.018+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae34062900 con 0x7fae34106f10 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.019+0000 7fae30ff9640 1 -- 172.21.15.103:0/1043077780 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fae2c005020 con 0x7fae34106f10 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.020+0000 7fae30ff9640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fae1c0a2c10 0x7fae1c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.020+0000 7fae337fe640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fae1c0a2c10 0x7fae1c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.020+0000 7fae30ff9640 1 -- 172.21.15.103:0/1043077780 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fae2c0d02f0 con 0x7fae34106f10 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.020+0000 7fae30ff9640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] conn(0x7fae1c0adde0 0x7fae1c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.020+0000 7fae30ff9640 1 -- 172.21.15.103:0/1043077780 --> [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae1c0b0950 con 0x7fae1c0adde0 2024-09-05T02:01:30.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.020+0000 7fae30ff9640 1 -- 172.21.15.103:0/1043077780 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fae2c0d0740 con 0x7fae34106f10 2024-09-05T02:01:30.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.021+0000 7fae32ffd640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] conn(0x7fae1c0adde0 0x7fae1c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.021+0000 7fae337fe640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fae1c0a2c10 0x7fae1c0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae24009b50 tx=0x7fae24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:30.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.021+0000 7fae32ffd640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] conn(0x7fae1c0adde0 0x7fae1c0b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:30.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.022+0000 7fae30ff9640 1 -- 172.21.15.103:0/1043077780 <== osd.6 v2:172.21.15.103:6818/2808952554 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fae1c0b0950 con 0x7fae1c0adde0 2024-09-05T02:01:30.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.035+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 --> [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fae34062d00 con 0x7fae1c0adde0 2024-09-05T02:01:30.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.035+0000 7fae30ff9640 1 -- 172.21.15.103:0/1043077780 <== osd.6 v2:172.21.15.103:6818/2808952554 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fae34062d00 con 0x7fae1c0adde0 2024-09-05T02:01:30.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] conn(0x7fae1c0adde0 msgr2=0x7fae1c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] conn(0x7fae1c0adde0 0x7fae1c0b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fae1c0a2c10 msgr2=0x7fae1c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fae1c0a2c10 0x7fae1c0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae24009b50 tx=0x7fae24005f90 comp rx=0 tx=0).stop 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae34106f10 msgr2=0x7fae3419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 --2- 172.21.15.103:0/1043077780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae34106f10 0x7fae3419cc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00eb10 tx=0x7fae2c00c2d0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae33fff640 1 -- 172.21.15.103:0/1043077780 reap_dead start 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 shutdown_connections 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 >> 172.21.15.103:0/1043077780 conn(0x7fae340fd8c0 msgr2=0x7fae340fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 shutdown_connections 2024-09-05T02:01:30.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.036+0000 7fae39bd9640 1 -- 172.21.15.103:0/1043077780 wait complete. 2024-09-05T02:01:30.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.950+0000 7f3740719640 1 -- 172.21.15.103:0/3940542663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373810c180 msgr2=0x7f373810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.950+0000 7f3740719640 1 --2- 172.21.15.103:0/3940542663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373810c180 0x7f373810e600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3734009900 tx=0x7f373402a480 comp rx=0 tx=0).stop 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.950+0000 7f3740719640 1 -- 172.21.15.103:0/3940542663 shutdown_connections 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.950+0000 7f3740719640 1 --2- 172.21.15.103:0/3940542663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373810c180 0x7f373810e600 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.950+0000 7f3740719640 1 --2- 172.21.15.103:0/3940542663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37381036c0 0x7f373810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.950+0000 7f3740719640 1 --2- 172.21.15.103:0/3940542663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3738102da0 0x7f3738103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.950+0000 7f3740719640 1 -- 172.21.15.103:0/3940542663 >> 172.21.15.103:0/3940542663 conn(0x7f37380fc7a0 msgr2=0x7f37380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.950+0000 7f3740719640 1 -- 172.21.15.103:0/3940542663 shutdown_connections 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.950+0000 7f3740719640 1 -- 172.21.15.103:0/3940542663 wait complete. 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f3740719640 1 Processor -- start 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f3740719640 1 -- start start 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f3740719640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3738102da0 0x7f3738100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f3740719640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37381036c0 0x7f3738100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f3740719640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373810c180 0x7f3738101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f3740719640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3738111e70 con 0x7f373810c180 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f3740719640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3738111cf0 con 0x7f37381036c0 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f3740719640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3738111ff0 con 0x7f3738102da0 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373ecb6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373810c180 0x7f3738101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373ecb6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373810c180 0x7f3738101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47406/0 (socket says 172.21.15.103:47406) 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373ecb6640 1 -- 172.21.15.103:0/3075465833 learned_addr learned my addr 172.21.15.103:0/3075465833 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373e4b5640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3738102da0 0x7f3738100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373dcb4640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37381036c0 0x7f3738100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373ecb6640 1 -- 172.21.15.103:0/3075465833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3738102da0 msgr2=0x7f3738100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373ecb6640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3738102da0 0x7f3738100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373ecb6640 1 -- 172.21.15.103:0/3075465833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37381036c0 msgr2=0x7f3738100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373ecb6640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37381036c0 0x7f3738100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.951+0000 7f373ecb6640 1 -- 172.21.15.103:0/3075465833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37381a6830 con 0x7f373810c180 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.952+0000 7f373ecb6640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373810c180 0x7f3738101480 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3734009230 tx=0x7f3734004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.952+0000 7f372f7fe640 1 -- 172.21.15.103:0/3075465833 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373403f030 con 0x7f373810c180 2024-09-05T02:01:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.952+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37381a6ac0 con 0x7f373810c180 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.952+0000 7f372f7fe640 1 -- 172.21.15.103:0/3075465833 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3734003aa0 con 0x7f373810c180 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.952+0000 7f372f7fe640 1 -- 172.21.15.103:0/3075465833 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734035070 con 0x7f373810c180 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.952+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37381a6ff0 con 0x7f373810c180 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.953+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3738062900 con 0x7f373810c180 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.954+0000 7f372f7fe640 1 -- 172.21.15.103:0/3075465833 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3734042030 con 0x7f373810c180 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.954+0000 7f372f7fe640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37240a2c10 0x7f37240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.955+0000 7f373e4b5640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37240a2c10 0x7f37240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.955+0000 7f372f7fe640 1 -- 172.21.15.103:0/3075465833 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f37340f0050 con 0x7f373810c180 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.955+0000 7f372f7fe640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] conn(0x7f37240adde0 0x7f37240b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.955+0000 7f372f7fe640 1 -- 172.21.15.103:0/3075465833 --> [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37240b0950 con 0x7f37240adde0 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.955+0000 7f372f7fe640 1 -- 172.21.15.103:0/3075465833 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f37340f04a0 con 0x7f373810c180 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.955+0000 7f373e4b5640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37240a2c10 0x7f37240a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f373000a400 tx=0x7f373000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.955+0000 7f373dcb4640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] conn(0x7f37240adde0 0x7f37240b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.956+0000 7f373dcb4640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] conn(0x7f37240adde0 0x7f37240b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:30.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.956+0000 7f372f7fe640 1 -- 172.21.15.103:0/3075465833 <== osd.2 v2:172.21.15.167:6800/945108637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37240b0950 con 0x7f37240adde0 2024-09-05T02:01:30.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.970+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 --> [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3738062d00 con 0x7f37240adde0 2024-09-05T02:01:30.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.970+0000 7f372f7fe640 1 -- 172.21.15.103:0/3075465833 <== osd.2 v2:172.21.15.167:6800/945108637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3738062d00 con 0x7f37240adde0 2024-09-05T02:01:30.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 >> [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] conn(0x7f37240adde0 msgr2=0x7f37240b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] conn(0x7f37240adde0 0x7f37240b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:30.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37240a2c10 msgr2=0x7f37240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37240a2c10 0x7f37240a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f373000a400 tx=0x7f373000d040 comp rx=0 tx=0).stop 2024-09-05T02:01:30.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373810c180 msgr2=0x7f3738101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:30.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 --2- 172.21.15.103:0/3075465833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373810c180 0x7f3738101480 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3734009230 tx=0x7f3734004270 comp rx=0 tx=0).stop 2024-09-05T02:01:30.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f373ecb6640 1 -- 172.21.15.103:0/3075465833 reap_dead start 2024-09-05T02:01:30.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 shutdown_connections 2024-09-05T02:01:30.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 >> 172.21.15.103:0/3075465833 conn(0x7f37380fc7a0 msgr2=0x7f37380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:30.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 shutdown_connections 2024-09-05T02:01:30.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:30.971+0000 7f3740719640 1 -- 172.21.15.103:0/3075465833 wait complete. 2024-09-05T02:01:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:31 smithi103 ceph-mon[30902]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-05T02:01:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:31 smithi103 ceph-mon[30902]: Cluster is now healthy 2024-09-05T02:01:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:31 smithi167 ceph-mon[35122]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-05T02:01:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:31 smithi167 ceph-mon[35122]: Cluster is now healthy 2024-09-05T02:01:31.648 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:31 smithi139 ceph-mon[34262]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-05T02:01:31.648 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:31 smithi139 ceph-mon[34262]: Cluster is now healthy 2024-09-05T02:01:32.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.149+0000 7f858c8dc640 1 -- 172.21.15.103:0/393888529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102ef0 msgr2=0x7f858810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.149+0000 7f858c8dc640 1 --2- 172.21.15.103:0/393888529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102ef0 0x7f858810da40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8580009ca0 tx=0x7f858002a950 comp rx=0 tx=0).stop 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.149+0000 7f858c8dc640 1 -- 172.21.15.103:0/393888529 shutdown_connections 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.149+0000 7f858c8dc640 1 --2- 172.21.15.103:0/393888529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102ef0 0x7f858810da40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.149+0000 7f858c8dc640 1 --2- 172.21.15.103:0/393888529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8588102570 0x7f85881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.149+0000 7f858c8dc640 1 --2- 172.21.15.103:0/393888529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588100e20 0x7f8588101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.149+0000 7f858c8dc640 1 -- 172.21.15.103:0/393888529 >> 172.21.15.103:0/393888529 conn(0x7f85880fc7d0 msgr2=0x7f85880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.149+0000 7f858c8dc640 1 -- 172.21.15.103:0/393888529 shutdown_connections 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.149+0000 7f858c8dc640 1 -- 172.21.15.103:0/393888529 wait complete. 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858c8dc640 1 Processor -- start 2024-09-05T02:01:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858c8dc640 1 -- start start 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858c8dc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588100e20 0x7f8588198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858c8dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8588102570 0x7f8588198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858c8dc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588102ef0 0x7f858819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858c8dc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8588110ed0 con 0x7f8588100e20 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858c8dc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8588110d50 con 0x7f8588102570 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858c8dc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8588111050 con 0x7f8588102ef0 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588100e20 0x7f8588198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588100e20 0x7f8588198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47414/0 (socket says 172.21.15.103:47414) 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858659c640 1 -- 172.21.15.103:0/2283902097 learned_addr learned my addr 172.21.15.103:0/2283902097 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f8586d9d640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588102ef0 0x7f858819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f8585d9b640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8588102570 0x7f8588198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858659c640 1 -- 172.21.15.103:0/2283902097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588102ef0 msgr2=0x7f858819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.150+0000 7f858659c640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588102ef0 0x7f858819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.151+0000 7f858659c640 1 -- 172.21.15.103:0/2283902097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8588102570 msgr2=0x7f8588198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.151+0000 7f858659c640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8588102570 0x7f8588198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.151+0000 7f858659c640 1 -- 172.21.15.103:0/2283902097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858819d4d0 con 0x7f8588100e20 2024-09-05T02:01:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.151+0000 7f858659c640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588100e20 0x7f8588198520 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f857800b4a0 tx=0x7f857800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.151+0000 7f85777fe640 1 -- 172.21.15.103:0/2283902097 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578015020 con 0x7f8588100e20 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.151+0000 7f85777fe640 1 -- 172.21.15.103:0/2283902097 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8578011ce0 con 0x7f8588100e20 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.151+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858819d7c0 con 0x7f8588100e20 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.151+0000 7f85777fe640 1 -- 172.21.15.103:0/2283902097 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578010dc0 con 0x7f8588100e20 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.151+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85881a50a0 con 0x7f8588100e20 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.152+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8588107220 con 0x7f8588100e20 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.153+0000 7f85777fe640 1 -- 172.21.15.103:0/2283902097 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8578018020 con 0x7f8588100e20 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.153+0000 7f85777fe640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f85700a2c10 0x7f85700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.154+0000 7f8585d9b640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f85700a2c10 0x7f85700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.154+0000 7f85777fe640 1 -- 172.21.15.103:0/2283902097 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f85780cf330 con 0x7f8588100e20 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.154+0000 7f85777fe640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] conn(0x7f85700adde0 0x7f85700b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.154+0000 7f85777fe640 1 -- 172.21.15.103:0/2283902097 --> [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8578018ec0 con 0x7f85700adde0 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.154+0000 7f85777fe640 1 -- 172.21.15.103:0/2283902097 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f85780cf810 con 0x7f8588100e20 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.154+0000 7f8585d9b640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f85700a2c10 0x7f85700a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85880fef70 tx=0x7f857c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.154+0000 7f8586d9d640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] conn(0x7f85700adde0 0x7f85700b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.155+0000 7f8586d9d640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] conn(0x7f85700adde0 0x7f85700b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.155+0000 7f85777fe640 1 -- 172.21.15.103:0/2283902097 <== osd.4 v2:172.21.15.139:6808/2565711850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8578018ec0 con 0x7f85700adde0 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.169+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 --> [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85881021b0 con 0x7f85700adde0 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.169+0000 7f85777fe640 1 -- 172.21.15.103:0/2283902097 <== osd.4 v2:172.21.15.139:6808/2565711850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f85881021b0 con 0x7f85700adde0 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 >> [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] conn(0x7f85700adde0 msgr2=0x7f85700b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] conn(0x7f85700adde0 0x7f85700b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f85700a2c10 msgr2=0x7f85700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f85700a2c10 0x7f85700a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85880fef70 tx=0x7f857c009110 comp rx=0 tx=0).stop 2024-09-05T02:01:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588100e20 msgr2=0x7f8588198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 --2- 172.21.15.103:0/2283902097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588100e20 0x7f8588198520 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f857800b4a0 tx=0x7f857800b8a0 comp rx=0 tx=0).stop 2024-09-05T02:01:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f8586d9d640 1 -- 172.21.15.103:0/2283902097 reap_dead start 2024-09-05T02:01:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 shutdown_connections 2024-09-05T02:01:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 >> 172.21.15.103:0/2283902097 conn(0x7f85880fc7d0 msgr2=0x7f8588069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 shutdown_connections 2024-09-05T02:01:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.170+0000 7f858c8dc640 1 -- 172.21.15.103:0/2283902097 wait complete. 2024-09-05T02:01:32.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:32 smithi103 ceph-mon[30902]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s; 13 B/s, 0 objects/s recovering 2024-09-05T02:01:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:32 smithi167 ceph-mon[35122]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s; 13 B/s, 0 objects/s recovering 2024-09-05T02:01:32.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:32 smithi139 ceph-mon[34262]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s; 13 B/s, 0 objects/s recovering 2024-09-05T02:01:32.989 INFO:teuthology.orchestra.run.smithi103.stdout:133143986241 2024-09-05T02:01:32.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-05T02:01:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.165+0000 7fea1b78a640 1 -- 172.21.15.103:0/481565011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea141036e0 msgr2=0x7fea14107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.165+0000 7fea1b78a640 1 --2- 172.21.15.103:0/481565011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea141036e0 0x7fea14107b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fea04009930 tx=0x7fea0402a420 comp rx=0 tx=0).stop 2024-09-05T02:01:33.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.165+0000 7fea1b78a640 1 -- 172.21.15.103:0/481565011 shutdown_connections 2024-09-05T02:01:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.165+0000 7fea1b78a640 1 --2- 172.21.15.103:0/481565011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea141082a0 0x7fea1410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.165+0000 7fea1b78a640 1 --2- 172.21.15.103:0/481565011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea141036e0 0x7fea14107b90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.165+0000 7fea1b78a640 1 --2- 172.21.15.103:0/481565011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea14102d30 0x7fea14103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.165+0000 7fea1b78a640 1 -- 172.21.15.103:0/481565011 >> 172.21.15.103:0/481565011 conn(0x7fea140fc7d0 msgr2=0x7fea140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.165+0000 7fea1b78a640 1 -- 172.21.15.103:0/481565011 shutdown_connections 2024-09-05T02:01:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.166+0000 7fea1b78a640 1 -- 172.21.15.103:0/481565011 wait complete. 2024-09-05T02:01:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.166+0000 7fea1b78a640 1 Processor -- start 2024-09-05T02:01:33.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.166+0000 7fea1b78a640 1 -- start start 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.166+0000 7fea1b78a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea14102d30 0x7fea14100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.166+0000 7fea1b78a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea141036e0 0x7fea14100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.166+0000 7fea1b78a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea141082a0 0x7fea14101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.166+0000 7fea1b78a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea14111f70 con 0x7fea141036e0 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.166+0000 7fea1b78a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fea14111df0 con 0x7fea141082a0 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.166+0000 7fea1b78a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea141120f0 con 0x7fea14102d30 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea18d25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea141036e0 0x7fea14100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea18d25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea141036e0 0x7fea14100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47440/0 (socket says 172.21.15.103:47440) 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea18d25640 1 -- 172.21.15.103:0/3208473330 learned_addr learned my addr 172.21.15.103:0/3208473330 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea19526640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea14102d30 0x7fea14100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea19d27640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea141082a0 0x7fea14101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:33.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea18d25640 1 -- 172.21.15.103:0/3208473330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea14102d30 msgr2=0x7fea14100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea18d25640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea14102d30 0x7fea14100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea18d25640 1 -- 172.21.15.103:0/3208473330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea141082a0 msgr2=0x7fea14101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea18d25640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea141082a0 0x7fea14101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea18d25640 1 -- 172.21.15.103:0/3208473330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea141a6970 con 0x7fea141036e0 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea18d25640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea141036e0 0x7fea14100ed0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fea04009230 tx=0x7fea04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea0a7fc640 1 -- 172.21.15.103:0/3208473330 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0403f030 con 0x7fea141036e0 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea141a6c00 con 0x7fea141036e0 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea0a7fc640 1 -- 172.21.15.103:0/3208473330 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea040035f0 con 0x7fea141036e0 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea0a7fc640 1 -- 172.21.15.103:0/3208473330 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04035070 con 0x7fea141036e0 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.167+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea141a7130 con 0x7fea141036e0 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.168+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea140628c0 con 0x7fea141036e0 2024-09-05T02:01:33.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.170+0000 7fea0a7fc640 1 -- 172.21.15.103:0/3208473330 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fea04003aa0 con 0x7fea141036e0 2024-09-05T02:01:33.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.170+0000 7fea0a7fc640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fea000a2c10 0x7fea000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:33.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.170+0000 7fea19526640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fea000a2c10 0x7fea000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:33.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.170+0000 7fea0a7fc640 1 -- 172.21.15.103:0/3208473330 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fea040f04d0 con 0x7fea141036e0 2024-09-05T02:01:33.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.170+0000 7fea0a7fc640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] conn(0x7fea000adde0 0x7fea000b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:33.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.170+0000 7fea0a7fc640 1 -- 172.21.15.103:0/3208473330 --> [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea000b0950 con 0x7fea000adde0 2024-09-05T02:01:33.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.170+0000 7fea0a7fc640 1 -- 172.21.15.103:0/3208473330 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fea040f0920 con 0x7fea141036e0 2024-09-05T02:01:33.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.171+0000 7fea19526640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fea000a2c10 0x7fea000a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea1000abe0 tx=0x7fea1000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:33.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.171+0000 7fea19d27640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] conn(0x7fea000adde0 0x7fea000b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:33.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.171+0000 7fea19d27640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] conn(0x7fea000adde0 0x7fea000b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:33.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.172+0000 7fea0a7fc640 1 -- 172.21.15.103:0/3208473330 <== osd.11 v2:172.21.15.167:6824/259212791 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea000b0950 con 0x7fea000adde0 2024-09-05T02:01:33.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.187+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 --> [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea14062cc0 con 0x7fea000adde0 2024-09-05T02:01:33.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.188+0000 7fea0a7fc640 1 -- 172.21.15.103:0/3208473330 <== osd.11 v2:172.21.15.167:6824/259212791 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea14062cc0 con 0x7fea000adde0 2024-09-05T02:01:33.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.188+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 >> [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] conn(0x7fea000adde0 msgr2=0x7fea000b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:33.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.188+0000 7fea1b78a640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] conn(0x7fea000adde0 0x7fea000b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:33.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.189+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fea000a2c10 msgr2=0x7fea000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:33.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.189+0000 7fea1b78a640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fea000a2c10 0x7fea000a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea1000abe0 tx=0x7fea1000a2d0 comp rx=0 tx=0).stop 2024-09-05T02:01:33.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.189+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea141036e0 msgr2=0x7fea14100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:33.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.189+0000 7fea1b78a640 1 --2- 172.21.15.103:0/3208473330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea141036e0 0x7fea14100ed0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fea04009230 tx=0x7fea04004270 comp rx=0 tx=0).stop 2024-09-05T02:01:33.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.189+0000 7fea19d27640 1 -- 172.21.15.103:0/3208473330 reap_dead start 2024-09-05T02:01:33.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.189+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 shutdown_connections 2024-09-05T02:01:33.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.189+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 >> 172.21.15.103:0/3208473330 conn(0x7fea140fc7d0 msgr2=0x7fea14105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:33.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.190+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 shutdown_connections 2024-09-05T02:01:33.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.190+0000 7fea1b78a640 1 -- 172.21.15.103:0/3208473330 wait complete. 2024-09-05T02:01:33.685 INFO:teuthology.orchestra.run.smithi103.stdout:236223201325 2024-09-05T02:01:33.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-05T02:01:34.295 INFO:teuthology.orchestra.run.smithi103.stdout:73014444106 2024-09-05T02:01:34.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-05T02:01:34.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.260+0000 7fe67ff4b640 1 -- 172.21.15.103:0/3292315089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe678102790 msgr2=0x7fe6781067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.260+0000 7fe67ff4b640 1 --2- 172.21.15.103:0/3292315089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe678102790 0x7fe6781067e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe668009900 tx=0x7fe66802a410 comp rx=0 tx=0).stop 2024-09-05T02:01:34.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.260+0000 7fe67ff4b640 1 -- 172.21.15.103:0/3292315089 shutdown_connections 2024-09-05T02:01:34.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.260+0000 7fe67ff4b640 1 --2- 172.21.15.103:0/3292315089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe678106f10 0x7fe67810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.260+0000 7fe67ff4b640 1 --2- 172.21.15.103:0/3292315089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe678102790 0x7fe6781067e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.260+0000 7fe67ff4b640 1 --2- 172.21.15.103:0/3292315089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe678101de0 0x7fe6781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.260+0000 7fe67ff4b640 1 -- 172.21.15.103:0/3292315089 >> 172.21.15.103:0/3292315089 conn(0x7fe6780fd8c0 msgr2=0x7fe6780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 -- 172.21.15.103:0/3292315089 shutdown_connections 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 -- 172.21.15.103:0/3292315089 wait complete. 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 Processor -- start 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 -- start start 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe678101de0 0x7fe6780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe678102790 0x7fe678074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe678106f10 0x7fe678074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe678111e20 con 0x7fe678102790 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe678111ca0 con 0x7fe678106f10 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.261+0000 7fe67ff4b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe678111fa0 con 0x7fe678101de0 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67d4e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe678102790 0x7fe678074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67d4e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe678102790 0x7fe678074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47420/0 (socket says 172.21.15.103:47420) 2024-09-05T02:01:34.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67d4e6640 1 -- 172.21.15.103:0/75937757 learned_addr learned my addr 172.21.15.103:0/75937757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:34.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67dce7640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe678101de0 0x7fe6780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67e4e8640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe678106f10 0x7fe678074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67d4e6640 1 -- 172.21.15.103:0/75937757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe678101de0 msgr2=0x7fe6780744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67d4e6640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe678101de0 0x7fe6780744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67d4e6640 1 -- 172.21.15.103:0/75937757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe678106f10 msgr2=0x7fe678074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67d4e6640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe678106f10 0x7fe678074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67d4e6640 1 -- 172.21.15.103:0/75937757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6781a6800 con 0x7fe678102790 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67dce7640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe678101de0 0x7fe6780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67d4e6640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe678102790 0x7fe678074a30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe668005940 tx=0x7fe6680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe66effd640 1 -- 172.21.15.103:0/75937757 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66803f030 con 0x7fe678102790 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6781a6a90 con 0x7fe678102790 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe66effd640 1 -- 172.21.15.103:0/75937757 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe668003cb0 con 0x7fe678102790 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.262+0000 7fe66effd640 1 -- 172.21.15.103:0/75937757 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668035070 con 0x7fe678102790 2024-09-05T02:01:34.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.263+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6781a6fc0 con 0x7fe678102790 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.264+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6780628c0 con 0x7fe678102790 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.265+0000 7fe66effd640 1 -- 172.21.15.103:0/75937757 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe668042030 con 0x7fe678102790 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.266+0000 7fe66effd640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6640a2c90 0x7fe6640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.266+0000 7fe67dce7640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6640a2c90 0x7fe6640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.266+0000 7fe66effd640 1 -- 172.21.15.103:0/75937757 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fe6680efea0 con 0x7fe678102790 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.266+0000 7fe66effd640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] conn(0x7fe6640ade60 0x7fe6640b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.266+0000 7fe66effd640 1 -- 172.21.15.103:0/75937757 --> [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6640b09d0 con 0x7fe6640ade60 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.266+0000 7fe66effd640 1 -- 172.21.15.103:0/75937757 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe6680f02f0 con 0x7fe678102790 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.266+0000 7fe67e4e8640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] conn(0x7fe6640ade60 0x7fe6640b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.266+0000 7fe67dce7640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6640a2c90 0x7fe6640a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe674009fc0 tx=0x7fe674005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.267+0000 7fe67e4e8640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] conn(0x7fe6640ade60 0x7fe6640b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.267+0000 7fe66effd640 1 -- 172.21.15.103:0/75937757 <== osd.0 v2:172.21.15.103:6802/2389557969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6640b09d0 con 0x7fe6640ade60 2024-09-05T02:01:34.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.281+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 --> [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe678062cf0 con 0x7fe6640ade60 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.281+0000 7fe66effd640 1 -- 172.21.15.103:0/75937757 <== osd.0 v2:172.21.15.103:6802/2389557969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe678062cf0 con 0x7fe6640ade60 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.281+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] conn(0x7fe6640ade60 msgr2=0x7fe6640b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.281+0000 7fe67ff4b640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] conn(0x7fe6640ade60 0x7fe6640b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.281+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6640a2c90 msgr2=0x7fe6640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.281+0000 7fe67ff4b640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6640a2c90 0x7fe6640a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe674009fc0 tx=0x7fe674005f90 comp rx=0 tx=0).stop 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.281+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe678102790 msgr2=0x7fe678074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.281+0000 7fe67ff4b640 1 --2- 172.21.15.103:0/75937757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe678102790 0x7fe678074a30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe668005940 tx=0x7fe6680042f0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.282+0000 7fe67e4e8640 1 -- 172.21.15.103:0/75937757 reap_dead start 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.282+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 shutdown_connections 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.282+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 >> 172.21.15.103:0/75937757 conn(0x7fe6780fd8c0 msgr2=0x7fe6780fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:34.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.282+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 shutdown_connections 2024-09-05T02:01:34.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:32.282+0000 7fe67ff4b640 1 -- 172.21.15.103:0/75937757 wait complete. 2024-09-05T02:01:34.630 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:34 smithi103 ceph-mon[30902]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s; 13 B/s, 0 objects/s recovering 2024-09-05T02:01:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:34 smithi167 ceph-mon[35122]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s; 13 B/s, 0 objects/s recovering 2024-09-05T02:01:34.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:34 smithi139 ceph-mon[34262]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s; 13 B/s, 0 objects/s recovering 2024-09-05T02:01:34.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.901+0000 7f914356c640 1 -- 172.21.15.103:0/1348350659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f913c1036e0 msgr2=0x7f913c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.901+0000 7f914356c640 1 --2- 172.21.15.103:0/1348350659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f913c1036e0 0x7f913c107b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f912c009930 tx=0x7f912c02a420 comp rx=0 tx=0).stop 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.901+0000 7f914356c640 1 -- 172.21.15.103:0/1348350659 shutdown_connections 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.901+0000 7f914356c640 1 --2- 172.21.15.103:0/1348350659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f913c1082a0 0x7f913c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.901+0000 7f914356c640 1 --2- 172.21.15.103:0/1348350659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f913c1036e0 0x7f913c107b90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.901+0000 7f914356c640 1 --2- 172.21.15.103:0/1348350659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c102d30 0x7f913c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.901+0000 7f914356c640 1 -- 172.21.15.103:0/1348350659 >> 172.21.15.103:0/1348350659 conn(0x7f913c0fc7d0 msgr2=0x7f913c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.901+0000 7f914356c640 1 -- 172.21.15.103:0/1348350659 shutdown_connections 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.901+0000 7f914356c640 1 -- 172.21.15.103:0/1348350659 wait complete. 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f914356c640 1 Processor -- start 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f914356c640 1 -- start start 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f914356c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f913c102d30 0x7f913c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f914356c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c1036e0 0x7f913c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f914356c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f913c1082a0 0x7f913c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f914356c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f913c111f50 con 0x7f913c1082a0 2024-09-05T02:01:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f914356c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f913c111dd0 con 0x7f913c102d30 2024-09-05T02:01:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f914356c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f913c1120d0 con 0x7f913c1036e0 2024-09-05T02:01:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9141b09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f913c1082a0 0x7f913c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9141b09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f913c1082a0 0x7f913c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47448/0 (socket says 172.21.15.103:47448) 2024-09-05T02:01:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9141b09640 1 -- 172.21.15.103:0/3466832612 learned_addr learned my addr 172.21.15.103:0/3466832612 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9141308640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f913c102d30 0x7f913c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9140b07640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c1036e0 0x7f913c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9141b09640 1 -- 172.21.15.103:0/3466832612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c1036e0 msgr2=0x7f913c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9141b09640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c1036e0 0x7f913c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9141b09640 1 -- 172.21.15.103:0/3466832612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f913c102d30 msgr2=0x7f913c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9141b09640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f913c102d30 0x7f913c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.902+0000 7f9141b09640 1 -- 172.21.15.103:0/3466832612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c19d4e0 con 0x7f913c1082a0 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.903+0000 7f9140b07640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c1036e0 0x7f913c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.903+0000 7f9141b09640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f913c1082a0 0x7f913c19ce00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f913400eaa0 tx=0x7f913400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.903+0000 7f91327fc640 1 -- 172.21.15.103:0/3466832612 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913400cdb0 con 0x7f913c1082a0 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.903+0000 7f91327fc640 1 -- 172.21.15.103:0/3466832612 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9134005020 con 0x7f913c1082a0 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.903+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c19d7d0 con 0x7f913c1082a0 2024-09-05T02:01:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.903+0000 7f91327fc640 1 -- 172.21.15.103:0/3466832612 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91340116d0 con 0x7f913c1082a0 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.903+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c1a50b0 con 0x7f913c1082a0 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.904+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f913c062940 con 0x7f913c1082a0 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.905+0000 7f91327fc640 1 -- 172.21.15.103:0/3466832612 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f91340051d0 con 0x7f913c1082a0 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.905+0000 7f91327fc640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91280a2c90 0x7f91280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.906+0000 7f9141308640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91280a2c90 0x7f91280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.906+0000 7f91327fc640 1 -- 172.21.15.103:0/3466832612 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f91340cfa00 con 0x7f913c1082a0 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.906+0000 7f91327fc640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] conn(0x7f91280ade60 0x7f91280b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.906+0000 7f91327fc640 1 -- 172.21.15.103:0/3466832612 --> [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9134014ec0 con 0x7f91280ade60 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.906+0000 7f91327fc640 1 -- 172.21.15.103:0/3466832612 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f91340cfe50 con 0x7f913c1082a0 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.906+0000 7f9141308640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91280a2c90 0x7f91280a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9138005fd0 tx=0x7f913800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:34.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.906+0000 7f9140b07640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] conn(0x7f91280ade60 0x7f91280b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.907+0000 7f9140b07640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] conn(0x7f91280ade60 0x7f91280b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.908+0000 7f91327fc640 1 -- 172.21.15.103:0/3466832612 <== osd.7 v2:172.21.15.139:6816/3442676613 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9134014ec0 con 0x7f91280ade60 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.921+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 --> [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f913c062dc0 con 0x7f91280ade60 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.921+0000 7f91327fc640 1 -- 172.21.15.103:0/3466832612 <== osd.7 v2:172.21.15.139:6816/3442676613 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f913c062dc0 con 0x7f91280ade60 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 >> [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] conn(0x7f91280ade60 msgr2=0x7f91280b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] conn(0x7f91280ade60 0x7f91280b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91280a2c90 msgr2=0x7f91280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91280a2c90 0x7f91280a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9138005fd0 tx=0x7f913800b390 comp rx=0 tx=0).stop 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f913c1082a0 msgr2=0x7f913c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 --2- 172.21.15.103:0/3466832612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f913c1082a0 0x7f913c19ce00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f913400eaa0 tx=0x7f913400ef60 comp rx=0 tx=0).stop 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f9141b09640 1 -- 172.21.15.103:0/3466832612 reap_dead start 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 shutdown_connections 2024-09-05T02:01:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 >> 172.21.15.103:0/3466832612 conn(0x7f913c0fc7d0 msgr2=0x7f913c105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 shutdown_connections 2024-09-05T02:01:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:33.922+0000 7f914356c640 1 -- 172.21.15.103:0/3466832612 wait complete. 2024-09-05T02:01:35.557 INFO:teuthology.orchestra.run.smithi103.stdout:206158430257 2024-09-05T02:01:35.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-05T02:01:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:184683593783 2024-09-05T02:01:36.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-05T02:01:36.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:36 smithi103 ceph-mon[30902]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-05T02:01:36.550 INFO:teuthology.orchestra.run.smithi103.stdout:317827579933 2024-09-05T02:01:36.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-05T02:01:36.610 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:36.610 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:36 smithi167 ceph-mon[35122]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-05T02:01:36.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:36 smithi139 ceph-mon[34262]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-05T02:01:37.082 INFO:teuthology.orchestra.run.smithi103.stdout:34359738452 2024-09-05T02:01:37.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-05T02:01:37.353 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:37.362 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:37.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.549+0000 7f12138d1640 1 -- 172.21.15.103:0/3854733644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 msgr2=0x7f120c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:37.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.549+0000 7f12138d1640 1 --2- 172.21.15.103:0/3854733644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 0x7f120c10d950 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009cd0 tx=0x7f11fc02a8e0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.549+0000 7f12138d1640 1 -- 172.21.15.103:0/3854733644 shutdown_connections 2024-09-05T02:01:37.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.549+0000 7f12138d1640 1 --2- 172.21.15.103:0/3854733644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 0x7f120c10d950 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.549+0000 7f12138d1640 1 --2- 172.21.15.103:0/3854733644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f120c100a80 0x7f120c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.549+0000 7f12138d1640 1 --2- 172.21.15.103:0/3854733644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120c106a80 0x7f120c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.549+0000 7f12138d1640 1 -- 172.21.15.103:0/3854733644 >> 172.21.15.103:0/3854733644 conn(0x7f120c0fc7a0 msgr2=0x7f120c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:37.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.549+0000 7f12138d1640 1 -- 172.21.15.103:0/3854733644 shutdown_connections 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.549+0000 7f12138d1640 1 -- 172.21.15.103:0/3854733644 wait complete. 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f12138d1640 1 Processor -- start 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f12138d1640 1 -- start start 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f12138d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f120c100a80 0x7f120c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f12138d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 0x7f120c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f12138d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120c106a80 0x7f120c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f12138d1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f120c110d50 con 0x7f120c101420 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f12138d1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f120c110bd0 con 0x7f120c100a80 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f12138d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f120c110ed0 con 0x7f120c106a80 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1210e6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 0x7f120c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:37.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1210e6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 0x7f120c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47454/0 (socket says 172.21.15.103:47454) 2024-09-05T02:01:37.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1210e6c640 1 -- 172.21.15.103:0/3830764113 learned_addr learned my addr 172.21.15.103:0/3830764113 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:37.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1211e6e640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120c106a80 0x7f120c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:37.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f121166d640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f120c100a80 0x7f120c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:37.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1210e6c640 1 -- 172.21.15.103:0/3830764113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120c106a80 msgr2=0x7f120c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:37.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1210e6c640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120c106a80 0x7f120c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1210e6c640 1 -- 172.21.15.103:0/3830764113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f120c100a80 msgr2=0x7f120c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:37.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1210e6c640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f120c100a80 0x7f120c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1210e6c640 1 -- 172.21.15.103:0/3830764113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c19d3d0 con 0x7f120c101420 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f121166d640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f120c100a80 0x7f120c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.550+0000 7f1210e6c640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 0x7f120c198960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f120000d950 tx=0x7f120000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.551+0000 7f120a7fc640 1 -- 172.21.15.103:0/3830764113 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200017070 con 0x7f120c101420 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.551+0000 7f120a7fc640 1 -- 172.21.15.103:0/3830764113 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1200009c30 con 0x7f120c101420 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.551+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c19d6c0 con 0x7f120c101420 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.551+0000 7f120a7fc640 1 -- 172.21.15.103:0/3830764113 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120000f620 con 0x7f120c101420 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.551+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c1a4fa0 con 0x7f120c101420 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.552+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f120c100ee0 con 0x7f120c101420 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.553+0000 7f120a7fc640 1 -- 172.21.15.103:0/3830764113 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f12000109a0 con 0x7f120c101420 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.553+0000 7f120a7fc640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f11f40a2c10 0x7f11f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.554+0000 7f121166d640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f11f40a2c10 0x7f11f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.554+0000 7f120a7fc640 1 -- 172.21.15.103:0/3830764113 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f12000cf290 con 0x7f120c101420 2024-09-05T02:01:37.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.554+0000 7f121166d640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f11f40a2c10 0x7f11f40a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1204006fd0 tx=0x7f1204008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:37.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.558+0000 7f120a7fc640 1 -- 172.21.15.103:0/3830764113 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1200009de0 con 0x7f120c101420 2024-09-05T02:01:37.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.732+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f120c10a080 con 0x7f120c101420 2024-09-05T02:01:37.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.732+0000 7f120a7fc640 1 -- 172.21.15.103:0/3830764113 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f120009bcd0 con 0x7f120c101420 2024-09-05T02:01:37.733 INFO:teuthology.orchestra.run.smithi103.stdout:154618822716 2024-09-05T02:01:37.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.734+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f11f40a2c10 msgr2=0x7f11f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:37.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.734+0000 7f12138d1640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f11f40a2c10 0x7f11f40a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1204006fd0 tx=0x7f1204008040 comp rx=0 tx=0).stop 2024-09-05T02:01:37.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.734+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 msgr2=0x7f120c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:37.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.734+0000 7f12138d1640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 0x7f120c198960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f120000d950 tx=0x7f120000de10 comp rx=0 tx=0).stop 2024-09-05T02:01:37.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.735+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 shutdown_connections 2024-09-05T02:01:37.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.735+0000 7f12138d1640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f11f40a2c10 0x7f11f40a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.735+0000 7f12138d1640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120c106a80 0x7f120c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.735+0000 7f12138d1640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c101420 0x7f120c198960 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.735+0000 7f12138d1640 1 --2- 172.21.15.103:0/3830764113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f120c100a80 0x7f120c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:37.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.735+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 >> 172.21.15.103:0/3830764113 conn(0x7f120c0fc7a0 msgr2=0x7f120c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:37.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.735+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 shutdown_connections 2024-09-05T02:01:37.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:37.735+0000 7f12138d1640 1 -- 172.21.15.103:0/3830764113 wait complete. 2024-09-05T02:01:38.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.221+0000 7fe8015d9640 1 -- 172.21.15.103:0/3833014225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc100d00 msgr2=0x7fe7fc106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:38.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.221+0000 7fe8015d9640 1 --2- 172.21.15.103:0/3833014225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc100d00 0x7fe7fc106840 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009930 tx=0x7fe7f002a420 comp rx=0 tx=0).stop 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.221+0000 7fe8015d9640 1 -- 172.21.15.103:0/3833014225 shutdown_connections 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.221+0000 7fe8015d9640 1 --2- 172.21.15.103:0/3833014225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7fc106f50 0x7fe7fc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.221+0000 7fe8015d9640 1 --2- 172.21.15.103:0/3833014225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc100d00 0x7fe7fc106840 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.221+0000 7fe8015d9640 1 --2- 172.21.15.103:0/3833014225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7fc100350 0x7fe7fc100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.221+0000 7fe8015d9640 1 -- 172.21.15.103:0/3833014225 >> 172.21.15.103:0/3833014225 conn(0x7fe7fc077210 msgr2=0x7fe7fc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.221+0000 7fe8015d9640 1 -- 172.21.15.103:0/3833014225 shutdown_connections 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.221+0000 7fe8015d9640 1 -- 172.21.15.103:0/3833014225 wait complete. 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe8015d9640 1 Processor -- start 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe8015d9640 1 -- start start 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe8015d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7fc100350 0x7fe7fc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:38.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe8015d9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7fc100d00 0x7fe7fc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe8015d9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc106f50 0x7fe7fc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe8015d9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc111e50 con 0x7fe7fc106f50 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe8015d9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc111cd0 con 0x7fe7fc100350 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe8015d9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc111fd0 con 0x7fe7fc100d00 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc106f50 0x7fe7fc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc106f50 0x7fe7fc19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47480/0 (socket says 172.21.15.103:47480) 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe7fb7fe640 1 -- 172.21.15.103:0/2904812907 learned_addr learned my addr 172.21.15.103:0/2904812907 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe7fa7fc640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7fc100d00 0x7fe7fc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.222+0000 7fe7faffd640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7fc100350 0x7fe7fc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe7fb7fe640 1 -- 172.21.15.103:0/2904812907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7fc100d00 msgr2=0x7fe7fc1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe7fb7fe640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7fc100d00 0x7fe7fc1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:38.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe7fb7fe640 1 -- 172.21.15.103:0/2904812907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7fc100350 msgr2=0x7fe7fc198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe7fb7fe640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7fc100350 0x7fe7fc198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe7fb7fe640 1 -- 172.21.15.103:0/2904812907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc19d430 con 0x7fe7fc106f50 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe7fb7fe640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc106f50 0x7fe7fc19cd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009fd0 tx=0x7fe7f400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe7e3fff640 1 -- 172.21.15.103:0/2904812907 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4017070 con 0x7fe7fc106f50 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe7e3fff640 1 -- 172.21.15.103:0/2904812907 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f4003e20 con 0x7fe7fc106f50 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7fc19d720 con 0x7fe7fc106f50 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe7e3fff640 1 -- 172.21.15.103:0/2904812907 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4010460 con 0x7fe7fc106f50 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.223+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7fc1a5000 con 0x7fe7fc106f50 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.224+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7fc100730 con 0x7fe7fc106f50 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.225+0000 7fe7e3fff640 1 -- 172.21.15.103:0/2904812907 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe7f4003970 con 0x7fe7fc106f50 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.226+0000 7fe7e3fff640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe7e40a2c90 0x7fe7e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.226+0000 7fe7faffd640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe7e40a2c90 0x7fe7e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.226+0000 7fe7e3fff640 1 -- 172.21.15.103:0/2904812907 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fe7f40cf2b0 con 0x7fe7fc106f50 2024-09-05T02:01:38.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.226+0000 7fe7faffd640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe7e40a2c90 0x7fe7e40a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009b60 tx=0x7fe7ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:38.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.230+0000 7fe7e3fff640 1 -- 172.21.15.103:0/2904812907 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7f409bd50 con 0x7fe7fc106f50 2024-09-05T02:01:38.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:38 smithi103 ceph-mon[30902]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s; 10 B/s, 0 objects/s recovering 2024-09-05T02:01:38.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:38 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3830764113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T02:01:38.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.403+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe7fc10a120 con 0x7fe7fc106f50 2024-09-05T02:01:38.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.404+0000 7fe7e3fff640 1 -- 172.21.15.103:0/2904812907 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe7f409b650 con 0x7fe7fc106f50 2024-09-05T02:01:38.405 INFO:teuthology.orchestra.run.smithi103.stdout:55834574928 2024-09-05T02:01:38.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe7e40a2c90 msgr2=0x7fe7e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:38.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe7e40a2c90 0x7fe7e40a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009b60 tx=0x7fe7ec005f90 comp rx=0 tx=0).stop 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc106f50 msgr2=0x7fe7fc19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc106f50 0x7fe7fc19cd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009fd0 tx=0x7fe7f400b260 comp rx=0 tx=0).stop 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 shutdown_connections 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7fc106f50 0x7fe7fc19cd50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe7e40a2c90 0x7fe7e40a5150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7fc100d00 0x7fe7fc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 --2- 172.21.15.103:0/2904812907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7fc100350 0x7fe7fc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 >> 172.21.15.103:0/2904812907 conn(0x7fe7fc077210 msgr2=0x7fe7fc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 shutdown_connections 2024-09-05T02:01:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:38.406+0000 7fe8015d9640 1 -- 172.21.15.103:0/2904812907 wait complete. 2024-09-05T02:01:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:38 smithi167 ceph-mon[35122]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s; 10 B/s, 0 objects/s recovering 2024-09-05T02:01:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:38 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3830764113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T02:01:38.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:38 smithi139 ceph-mon[34262]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s; 10 B/s, 0 objects/s recovering 2024-09-05T02:01:38.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:38 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3830764113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T02:01:38.771 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822713 got 154618822716 for osd.5 2024-09-05T02:01:38.772 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:39.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:39 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2904812907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T02:01:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:39.443 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574928 for osd.1 2024-09-05T02:01:39.443 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:39.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:39 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2904812907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T02:01:39.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:39 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2904812907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T02:01:39.668 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:39.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.885+0000 7f2d21fa6640 1 -- 172.21.15.103:0/2170707733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c10c170 msgr2=0x7f2d1c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:39.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.885+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/2170707733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c10c170 0x7f2d1c10e610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009ca0 tx=0x7f2d1402a950 comp rx=0 tx=0).stop 2024-09-05T02:01:39.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.885+0000 7f2d21fa6640 1 -- 172.21.15.103:0/2170707733 shutdown_connections 2024-09-05T02:01:39.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.885+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/2170707733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c10c170 0x7f2d1c10e610 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:39.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.885+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/2170707733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d1c103660 0x7f2d1c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:39.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.885+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/2170707733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c102d40 0x7f2d1c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:39.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.885+0000 7f2d21fa6640 1 -- 172.21.15.103:0/2170707733 >> 172.21.15.103:0/2170707733 conn(0x7f2d1c0fc7a0 msgr2=0x7f2d1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:39.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.885+0000 7f2d21fa6640 1 -- 172.21.15.103:0/2170707733 shutdown_connections 2024-09-05T02:01:39.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.885+0000 7f2d21fa6640 1 -- 172.21.15.103:0/2170707733 wait complete. 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.886+0000 7f2d21fa6640 1 Processor -- start 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.886+0000 7f2d21fa6640 1 -- start start 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.886+0000 7f2d21fa6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c102d40 0x7f2d1c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.886+0000 7f2d21fa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d1c103660 0x7f2d1c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d21fa6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c10c170 0x7f2d1c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d21fa6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c111e20 con 0x7f2d1c102d40 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d21fa6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c111ca0 con 0x7f2d1c103660 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d21fa6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c111fa0 con 0x7f2d1c10c170 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c102d40 0x7f2d1c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c102d40 0x7f2d1c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45100/0 (socket says 172.21.15.103:45100) 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1b7fe640 1 -- 172.21.15.103:0/3544712388 learned_addr learned my addr 172.21.15.103:0/3544712388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:39.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1bfff640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c10c170 0x7f2d1c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1b7fe640 1 -- 172.21.15.103:0/3544712388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c10c170 msgr2=0x7f2d1c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1b7fe640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c10c170 0x7f2d1c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1b7fe640 1 -- 172.21.15.103:0/3544712388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d1c103660 msgr2=0x7f2d1c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1b7fe640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d1c103660 0x7f2d1c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1b7fe640 1 -- 172.21.15.103:0/3544712388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c19d2f0 con 0x7f2d1c102d40 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1bfff640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c10c170 0x7f2d1c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d1b7fe640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c102d40 0x7f2d1c198340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e9f0 tx=0x7f2d0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d18ff9640 1 -- 172.21.15.103:0/3544712388 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00cd80 con 0x7f2d1c102d40 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d18ff9640 1 -- 172.21.15.103:0/3544712388 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d0c0054d0 con 0x7f2d1c102d40 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c19d5e0 con 0x7f2d1c102d40 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d18ff9640 1 -- 172.21.15.103:0/3544712388 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c010490 con 0x7f2d1c102d40 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.887+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c1a4ec0 con 0x7f2d1c102d40 2024-09-05T02:01:39.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.888+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c103120 con 0x7f2d1c102d40 2024-09-05T02:01:39.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.889+0000 7f2d18ff9640 1 -- 172.21.15.103:0/3544712388 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2d0c005020 con 0x7f2d1c102d40 2024-09-05T02:01:39.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.890+0000 7f2d18ff9640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d040a2c10 0x7f2d040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:39.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.890+0000 7f2d1affd640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d040a2c10 0x7f2d040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:39.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.890+0000 7f2d18ff9640 1 -- 172.21.15.103:0/3544712388 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f2d0c0cfa10 con 0x7f2d1c102d40 2024-09-05T02:01:39.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.891+0000 7f2d1affd640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d040a2c10 0x7f2d040a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d10005fd0 tx=0x7f2d10005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:39.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:39.894+0000 7f2d18ff9640 1 -- 172.21.15.103:0/3544712388 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d0c09c4b0 con 0x7f2d1c102d40 2024-09-05T02:01:39.926 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:39.927 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:40.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.080+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2d1c1996a0 con 0x7f2d1c102d40 2024-09-05T02:01:40.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.080+0000 7f2d18ff9640 1 -- 172.21.15.103:0/3544712388 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2d0c09bdb0 con 0x7f2d1c102d40 2024-09-05T02:01:40.081 INFO:teuthology.orchestra.run.smithi103.stdout:103079215175 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d040a2c10 msgr2=0x7f2d040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d040a2c10 0x7f2d040a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d10005fd0 tx=0x7f2d10005d80 comp rx=0 tx=0).stop 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c102d40 msgr2=0x7f2d1c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c102d40 0x7f2d1c198340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e9f0 tx=0x7f2d0c00edf0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 shutdown_connections 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d040a2c10 0x7f2d040a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c10c170 0x7f2d1c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d1c103660 0x7f2d1c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 --2- 172.21.15.103:0/3544712388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d1c102d40 0x7f2d1c198340 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 >> 172.21.15.103:0/3544712388 conn(0x7f2d1c0fc7a0 msgr2=0x7f2d1c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 shutdown_connections 2024-09-05T02:01:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.082+0000 7f2d21fa6640 1 -- 172.21.15.103:0/3544712388 wait complete. 2024-09-05T02:01:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:40 smithi103 ceph-mon[30902]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-05T02:01:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:40 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3544712388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T02:01:40.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 -- 172.21.15.103:0/550516826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0100e20 msgr2=0x7fabb0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:40.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 --2- 172.21.15.103:0/550516826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0100e20 0x7fabb0101200 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faba0009900 tx=0x7faba002a480 comp rx=0 tx=0).stop 2024-09-05T02:01:40.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 -- 172.21.15.103:0/550516826 shutdown_connections 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 --2- 172.21.15.103:0/550516826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb0102ef0 0x7fabb010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 --2- 172.21.15.103:0/550516826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb0102570 0x7fabb01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 --2- 172.21.15.103:0/550516826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0100e20 0x7fabb0101200 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 -- 172.21.15.103:0/550516826 >> 172.21.15.103:0/550516826 conn(0x7fabb00fc7d0 msgr2=0x7fabb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 -- 172.21.15.103:0/550516826 shutdown_connections 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 -- 172.21.15.103:0/550516826 wait complete. 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 Processor -- start 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.550+0000 7fabb538e640 1 -- start start 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabb538e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb0100e20 0x7fabb0196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabb538e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0102570 0x7fabb0196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabb538e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb0102ef0 0x7fabb019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabb538e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabb010fd30 con 0x7fabb0102570 2024-09-05T02:01:40.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabb538e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fabb010fbb0 con 0x7fabb0102ef0 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabb538e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabb010feb0 con 0x7fabb0100e20 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7faba7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0102570 0x7fabb0196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7faba7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0102570 0x7fabb0196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45128/0 (socket says 172.21.15.103:45128) 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7faba7fff640 1 -- 172.21.15.103:0/1896376555 learned_addr learned my addr 172.21.15.103:0/1896376555 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabaeffd640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb0100e20 0x7fabb0196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7faba7fff640 1 -- 172.21.15.103:0/1896376555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb0100e20 msgr2=0x7fabb0196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabaf7fe640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb0102ef0 0x7fabb019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7faba7fff640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb0100e20 0x7fabb0196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7faba7fff640 1 -- 172.21.15.103:0/1896376555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb0102ef0 msgr2=0x7fabb019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7faba7fff640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb0102ef0 0x7fabb019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7faba7fff640 1 -- 172.21.15.103:0/1896376555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb019b200 con 0x7fabb0102570 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabaeffd640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb0100e20 0x7fabb0196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabaf7fe640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb0102ef0 0x7fabb019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7faba7fff640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0102570 0x7fabb0196790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab98009db0 tx=0x7fab9800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabacff9640 1 -- 172.21.15.103:0/1896376555 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9800b020 con 0x7fabb0102570 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb019b4f0 con 0x7fabb0102570 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabacff9640 1 -- 172.21.15.103:0/1896376555 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab9800b1c0 con 0x7fabb0102570 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.551+0000 7fabacff9640 1 -- 172.21.15.103:0/1896376555 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab980176f0 con 0x7fabb0102570 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.552+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb01a2dd0 con 0x7fabb0102570 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.553+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb0101200 con 0x7fabb0102570 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.554+0000 7fabacff9640 1 -- 172.21.15.103:0/1896376555 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fab9800a440 con 0x7fabb0102570 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.554+0000 7fabacff9640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab9c0a2c10 0x7fab9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:40.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.554+0000 7fabaeffd640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab9c0a2c10 0x7fab9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:40.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.554+0000 7fabacff9640 1 -- 172.21.15.103:0/1896376555 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fab980d11a0 con 0x7fabb0102570 2024-09-05T02:01:40.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.555+0000 7fabaeffd640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab9c0a2c10 0x7fab9c0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faba0005ab0 tx=0x7faba0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:40.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.558+0000 7fabacff9640 1 -- 172.21.15.103:0/1896376555 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab98012070 con 0x7fabb0102570 2024-09-05T02:01:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:40 smithi167 ceph-mon[35122]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-05T02:01:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:40 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3544712388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T02:01:40.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:40 smithi139 ceph-mon[34262]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-05T02:01:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:40 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3544712388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T02:01:40.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.735+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fabb01090c0 con 0x7fabb0102570 2024-09-05T02:01:40.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.736+0000 7fabacff9640 1 -- 172.21.15.103:0/1896376555 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fab9809dc40 con 0x7fabb0102570 2024-09-05T02:01:40.737 INFO:teuthology.orchestra.run.smithi103.stdout:184683593785 2024-09-05T02:01:40.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab9c0a2c10 msgr2=0x7fab9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:40.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab9c0a2c10 0x7fab9c0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faba0005ab0 tx=0x7faba0005a20 comp rx=0 tx=0).stop 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0102570 msgr2=0x7fabb0196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0102570 0x7fabb0196790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab98009db0 tx=0x7fab9800f4a0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 shutdown_connections 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb0102ef0 0x7fabb019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb0102570 0x7fabb0196790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab9c0a2c10 0x7fab9c0a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 --2- 172.21.15.103:0/1896376555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb0100e20 0x7fabb0196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 >> 172.21.15.103:0/1896376555 conn(0x7fabb00fc7d0 msgr2=0x7fabb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 shutdown_connections 2024-09-05T02:01:40.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:40.738+0000 7fabb538e640 1 -- 172.21.15.103:0/1896376555 wait complete. 2024-09-05T02:01:41.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:41 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1896376555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T02:01:41.500 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593783 got 184683593785 for osd.6 2024-09-05T02:01:41.500 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:41.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:41 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1896376555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T02:01:41.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:41 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1896376555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T02:01:42.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 -- 172.21.15.103:0/1528900442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 msgr2=0x7f2d7c106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 --2- 172.21.15.103:0/1528900442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 0x7f2d7c106f80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009900 tx=0x7f2d6c02a410 comp rx=0 tx=0).stop 2024-09-05T02:01:42.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 -- 172.21.15.103:0/1528900442 shutdown_connections 2024-09-05T02:01:42.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 --2- 172.21.15.103:0/1528900442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d7c100a70 0x7f2d7c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 --2- 172.21.15.103:0/1528900442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7c1000d0 0x7f2d7c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 --2- 172.21.15.103:0/1528900442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 0x7f2d7c106f80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 -- 172.21.15.103:0/1528900442 >> 172.21.15.103:0/1528900442 conn(0x7f2d7c077210 msgr2=0x7f2d7c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:42.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 -- 172.21.15.103:0/1528900442 shutdown_connections 2024-09-05T02:01:42.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 -- 172.21.15.103:0/1528900442 wait complete. 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.003+0000 7f2d819a0640 1 Processor -- start 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d819a0640 1 -- start start 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d819a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7c1000d0 0x7f2d7c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d819a0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d7c100a70 0x7f2d7c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d819a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 0x7f2d7c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d819a0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c111fe0 con 0x7f2d7c106ba0 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d819a0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c111e60 con 0x7f2d7c1000d0 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d819a0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c112160 con 0x7f2d7c100a70 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d7b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 0x7f2d7c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d7b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 0x7f2d7c19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45150/0 (socket says 172.21.15.103:45150) 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d7b7fe640 1 -- 172.21.15.103:0/3324104509 learned_addr learned my addr 172.21.15.103:0/3324104509 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:42.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d7b7fe640 1 -- 172.21.15.103:0/3324104509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d7c100a70 msgr2=0x7f2d7c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d7b7fe640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d7c100a70 0x7f2d7c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d7b7fe640 1 -- 172.21.15.103:0/3324104509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7c1000d0 msgr2=0x7f2d7c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d7b7fe640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7c1000d0 0x7f2d7c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d7b7fe640 1 -- 172.21.15.103:0/3324104509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c19d460 con 0x7f2d7c106ba0 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d7b7fe640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 0x7f2d7c19cd80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400b970 tx=0x7f2d7400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.004+0000 7f2d8099e640 1 -- 172.21.15.103:0/3324104509 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74019070 con 0x7f2d7c106ba0 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.005+0000 7f2d8099e640 1 -- 172.21.15.103:0/3324104509 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d74014440 con 0x7f2d7c106ba0 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.005+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c19d750 con 0x7f2d7c106ba0 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.005+0000 7f2d8099e640 1 -- 172.21.15.103:0/3324104509 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d740075d0 con 0x7f2d7c106ba0 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.005+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c1a5030 con 0x7f2d7c106ba0 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.006+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7c100530 con 0x7f2d7c106ba0 2024-09-05T02:01:42.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.007+0000 7f2d8099e640 1 -- 172.21.15.103:0/3324104509 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2d7401c070 con 0x7f2d7c106ba0 2024-09-05T02:01:42.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.008+0000 7f2d8099e640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d680a2c90 0x7f2d680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:42.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.008+0000 7f2d7affd640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d680a2c90 0x7f2d680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:42.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.008+0000 7f2d8099e640 1 -- 172.21.15.103:0/3324104509 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f2d740cee20 con 0x7f2d7c106ba0 2024-09-05T02:01:42.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.008+0000 7f2d7affd640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d680a2c90 0x7f2d680a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c02a930 tx=0x7f2d6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:42.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.011+0000 7f2d8099e640 1 -- 172.21.15.103:0/3324104509 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d7409b9d0 con 0x7f2d7c106ba0 2024-09-05T02:01:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.181+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2d7c199810 con 0x7f2d7c106ba0 2024-09-05T02:01:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.181+0000 7f2d8099e640 1 -- 172.21.15.103:0/3324104509 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2d7c199810 con 0x7f2d7c106ba0 2024-09-05T02:01:42.183 INFO:teuthology.orchestra.run.smithi103.stdout:133143986243 2024-09-05T02:01:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d680a2c90 msgr2=0x7f2d680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d680a2c90 0x7f2d680a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c02a930 tx=0x7f2d6c000bc0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 msgr2=0x7f2d7c19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 0x7f2d7c19cd80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400b970 tx=0x7f2d7400be30 comp rx=0 tx=0).stop 2024-09-05T02:01:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 shutdown_connections 2024-09-05T02:01:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d7c106ba0 0x7f2d7c19cd80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2d680a2c90 0x7f2d680a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d7c100a70 0x7f2d7c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 --2- 172.21.15.103:0/3324104509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7c1000d0 0x7f2d7c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.183+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 >> 172.21.15.103:0/3324104509 conn(0x7f2d7c077210 msgr2=0x7f2d7c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.184+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 shutdown_connections 2024-09-05T02:01:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.184+0000 7f2d819a0640 1 -- 172.21.15.103:0/3324104509 wait complete. 2024-09-05T02:01:42.327 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:42 smithi103 ceph-mon[30902]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-05T02:01:42.327 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:42 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3324104509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T02:01:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:42 smithi167 ceph-mon[35122]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-05T02:01:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:42 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3324104509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T02:01:42.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:42 smithi139 ceph-mon[34262]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-05T02:01:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:42 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3324104509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T02:01:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.794+0000 7f28dafc0640 1 -- 172.21.15.103:0/1692329947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d41027a0 msgr2=0x7f28d41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.794+0000 7f28dafc0640 1 --2- 172.21.15.103:0/1692329947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d41027a0 0x7f28d41067f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009900 tx=0x7f28c002a410 comp rx=0 tx=0).stop 2024-09-05T02:01:42.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.794+0000 7f28dafc0640 1 -- 172.21.15.103:0/1692329947 shutdown_connections 2024-09-05T02:01:42.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.794+0000 7f28dafc0640 1 --2- 172.21.15.103:0/1692329947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d4106f20 0x7f28d410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.794+0000 7f28dafc0640 1 --2- 172.21.15.103:0/1692329947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d41027a0 0x7f28d41067f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.794+0000 7f28dafc0640 1 --2- 172.21.15.103:0/1692329947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4101df0 0x7f28d41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.794+0000 7f28dafc0640 1 -- 172.21.15.103:0/1692329947 >> 172.21.15.103:0/1692329947 conn(0x7f28d40fd8b0 msgr2=0x7f28d40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:42.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.794+0000 7f28dafc0640 1 -- 172.21.15.103:0/1692329947 shutdown_connections 2024-09-05T02:01:42.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.794+0000 7f28dafc0640 1 -- 172.21.15.103:0/1692329947 wait complete. 2024-09-05T02:01:42.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28dafc0640 1 Processor -- start 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28dafc0640 1 -- start start 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28dafc0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d4101df0 0x7f28d4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28dafc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d41027a0 0x7f28d4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28dafc0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4106f20 0x7f28d419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28dafc0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4111e50 con 0x7f28d4101df0 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28dafc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4111cd0 con 0x7f28d41027a0 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28dafc0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4111fd0 con 0x7f28d4106f20 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28d8d5c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d4101df0 0x7f28d4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28d8d5c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d4101df0 0x7f28d4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45178/0 (socket says 172.21.15.103:45178) 2024-09-05T02:01:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28d8d5c640 1 -- 172.21.15.103:0/2234176109 learned_addr learned my addr 172.21.15.103:0/2234176109 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:42.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28d8d5c640 1 -- 172.21.15.103:0/2234176109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4106f20 msgr2=0x7f28d419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28d8d5c640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4106f20 0x7f28d419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28d8d5c640 1 -- 172.21.15.103:0/2234176109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d41027a0 msgr2=0x7f28d4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28d8d5c640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d41027a0 0x7f28d4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28d8d5c640 1 -- 172.21.15.103:0/2234176109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d419d2b0 con 0x7f28d4101df0 2024-09-05T02:01:42.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.795+0000 7f28d8d5c640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d4101df0 0x7f28d4198300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c920 tx=0x7f28c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:42.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.796+0000 7f28d1ffb640 1 -- 172.21.15.103:0/2234176109 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8007c70 con 0x7f28d4101df0 2024-09-05T02:01:42.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.796+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d419d5a0 con 0x7f28d4101df0 2024-09-05T02:01:42.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.796+0000 7f28d1ffb640 1 -- 172.21.15.103:0/2234176109 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28c8007e10 con 0x7f28d4101df0 2024-09-05T02:01:42.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.796+0000 7f28d1ffb640 1 -- 172.21.15.103:0/2234176109 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8018820 con 0x7f28d4101df0 2024-09-05T02:01:42.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.796+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d41a4e80 con 0x7f28d4101df0 2024-09-05T02:01:42.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.797+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d41021d0 con 0x7f28d4101df0 2024-09-05T02:01:42.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.798+0000 7f28d1ffb640 1 -- 172.21.15.103:0/2234176109 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f28c80189c0 con 0x7f28d4101df0 2024-09-05T02:01:42.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.798+0000 7f28d1ffb640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28c40a2c10 0x7f28c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:42.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.799+0000 7f28d3fff640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28c40a2c10 0x7f28c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:42.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.799+0000 7f28d1ffb640 1 -- 172.21.15.103:0/2234176109 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f28c80cf750 con 0x7f28d4101df0 2024-09-05T02:01:42.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.799+0000 7f28d3fff640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28c40a2c10 0x7f28c40a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f28c00096c0 tx=0x7f28c0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:42.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.803+0000 7f28d1ffb640 1 -- 172.21.15.103:0/2234176109 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28c8014030 con 0x7f28d4101df0 2024-09-05T02:01:42.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.976+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f28d410a130 con 0x7f28d4101df0 2024-09-05T02:01:42.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.976+0000 7f28d1ffb640 1 -- 172.21.15.103:0/2234176109 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f28c809c1f0 con 0x7f28d4101df0 2024-09-05T02:01:42.979 INFO:teuthology.orchestra.run.smithi103.stdout:266287972394 2024-09-05T02:01:42.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28c40a2c10 msgr2=0x7f28c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28c40a2c10 0x7f28c40a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f28c00096c0 tx=0x7f28c0031400 comp rx=0 tx=0).stop 2024-09-05T02:01:42.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d4101df0 msgr2=0x7f28d4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:42.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d4101df0 0x7f28d4198300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c920 tx=0x7f28c800cde0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 shutdown_connections 2024-09-05T02:01:42.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f28c40a2c10 0x7f28c40a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4106f20 0x7f28d419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d41027a0 0x7f28d4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 --2- 172.21.15.103:0/2234176109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28d4101df0 0x7f28d4198300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:42.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 >> 172.21.15.103:0/2234176109 conn(0x7f28d40fd8b0 msgr2=0x7f28d40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:42.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 shutdown_connections 2024-09-05T02:01:42.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:42.978+0000 7f28dafc0640 1 -- 172.21.15.103:0/2234176109 wait complete. 2024-09-05T02:01:43.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:43 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2234176109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T02:01:43.525 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986241 got 133143986243 for osd.4 2024-09-05T02:01:43.525 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:43.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:43 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2234176109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T02:01:43.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:43 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2234176109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T02:01:43.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.694+0000 7fced36ac640 1 -- 172.21.15.103:0/4058601697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 msgr2=0x7fcecc103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:43.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.694+0000 7fced36ac640 1 --2- 172.21.15.103:0/4058601697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 0x7fcecc103180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc02a480 comp rx=0 tx=0).stop 2024-09-05T02:01:43.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.694+0000 7fced36ac640 1 -- 172.21.15.103:0/4058601697 shutdown_connections 2024-09-05T02:01:43.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.694+0000 7fced36ac640 1 --2- 172.21.15.103:0/4058601697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcecc10c180 0x7fcecc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.694+0000 7fced36ac640 1 --2- 172.21.15.103:0/4058601697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcecc1036c0 0x7fcecc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.694+0000 7fced36ac640 1 --2- 172.21.15.103:0/4058601697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 0x7fcecc103180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.694+0000 7fced36ac640 1 -- 172.21.15.103:0/4058601697 >> 172.21.15.103:0/4058601697 conn(0x7fcecc0fc7a0 msgr2=0x7fcecc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.694+0000 7fced36ac640 1 -- 172.21.15.103:0/4058601697 shutdown_connections 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.694+0000 7fced36ac640 1 -- 172.21.15.103:0/4058601697 wait complete. 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced36ac640 1 Processor -- start 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced36ac640 1 -- start start 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced36ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 0x7fcecc1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced36ac640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcecc1036c0 0x7fcecc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced36ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcecc10c180 0x7fcecc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced36ac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc111e10 con 0x7fcecc102da0 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced36ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc111c90 con 0x7fcecc10c180 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced36ac640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc111f90 con 0x7fcecc1036c0 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced1448640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 0x7fcecc1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced1448640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 0x7fcecc1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45206/0 (socket says 172.21.15.103:45206) 2024-09-05T02:01:43.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced1448640 1 -- 172.21.15.103:0/3412166384 learned_addr learned my addr 172.21.15.103:0/3412166384 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.695+0000 7fced1c49640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcecc10c180 0x7fcecc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fced0c47640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcecc1036c0 0x7fcecc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fced1448640 1 -- 172.21.15.103:0/3412166384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcecc1036c0 msgr2=0x7fcecc198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fced1448640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcecc1036c0 0x7fcecc198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fced1448640 1 -- 172.21.15.103:0/3412166384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcecc10c180 msgr2=0x7fcecc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fced1448640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcecc10c180 0x7fcecc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fced1448640 1 -- 172.21.15.103:0/3412166384 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc19d270 con 0x7fcecc102da0 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fced1448640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 0x7fcecc1982c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcebc006010 tx=0x7fcebc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fcec27fc640 1 -- 172.21.15.103:0/3412166384 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc03f030 con 0x7fcecc102da0 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fcec27fc640 1 -- 172.21.15.103:0/3412166384 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcebc0035f0 con 0x7fcecc102da0 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc19d500 con 0x7fcecc102da0 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fcec27fc640 1 -- 172.21.15.103:0/3412166384 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc035070 con 0x7fcecc102da0 2024-09-05T02:01:43.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.696+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc1a4e40 con 0x7fcecc102da0 2024-09-05T02:01:43.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.697+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcecc06b010 con 0x7fcecc102da0 2024-09-05T02:01:43.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.698+0000 7fcec27fc640 1 -- 172.21.15.103:0/3412166384 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcebc003b10 con 0x7fcecc102da0 2024-09-05T02:01:43.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.699+0000 7fcec27fc640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fceb80a2c10 0x7fceb80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:43.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.699+0000 7fced0c47640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fceb80a2c10 0x7fceb80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:43.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.699+0000 7fcec27fc640 1 -- 172.21.15.103:0/3412166384 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fcebc0f02c0 con 0x7fcecc102da0 2024-09-05T02:01:43.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.699+0000 7fced0c47640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fceb80a2c10 0x7fceb80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcec8005e10 tx=0x7fcec8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:43.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.702+0000 7fcec27fc640 1 -- 172.21.15.103:0/3412166384 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcebc0bcd60 con 0x7fcecc102da0 2024-09-05T02:01:43.739 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:43.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.879+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fcecc107220 con 0x7fcecc102da0 2024-09-05T02:01:43.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.879+0000 7fcec27fc640 1 -- 172.21.15.103:0/3412166384 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcebc0bc660 con 0x7fcecc102da0 2024-09-05T02:01:43.881 INFO:teuthology.orchestra.run.smithi103.stdout:287762808868 2024-09-05T02:01:43.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fceb80a2c10 msgr2=0x7fceb80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:43.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fceb80a2c10 0x7fceb80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcec8005e10 tx=0x7fcec8005da0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 msgr2=0x7fcecc1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:43.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 0x7fcecc1982c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcebc006010 tx=0x7fcebc004270 comp rx=0 tx=0).stop 2024-09-05T02:01:43.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 shutdown_connections 2024-09-05T02:01:43.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcecc10c180 0x7fcecc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fceb80a2c10 0x7fceb80a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcecc1036c0 0x7fcecc198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 --2- 172.21.15.103:0/3412166384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcecc102da0 0x7fcecc1982c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:43.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 >> 172.21.15.103:0/3412166384 conn(0x7fcecc0fc7a0 msgr2=0x7fcecc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:43.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 shutdown_connections 2024-09-05T02:01:43.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:43.881+0000 7fced36ac640 1 -- 172.21.15.103:0/3412166384 wait complete. 2024-09-05T02:01:44.382 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215172 got 103079215175 for osd.3 2024-09-05T02:01:44.382 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:44.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:44 smithi103 ceph-mon[30902]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-05T02:01:44.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:44 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3412166384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T02:01:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:44 smithi167 ceph-mon[35122]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-05T02:01:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:44 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3412166384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T02:01:44.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:44 smithi139 ceph-mon[34262]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-05T02:01:44.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:44 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3412166384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T02:01:44.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.958+0000 7f09e1488640 1 -- 172.21.15.103:0/3721479428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc101400 msgr2=0x7f09dc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:44.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.958+0000 7f09e1488640 1 --2- 172.21.15.103:0/3721479428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc101400 0x7f09dc10d930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009930 tx=0x7f09cc02a420 comp rx=0 tx=0).stop 2024-09-05T02:01:44.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.958+0000 7f09e1488640 1 -- 172.21.15.103:0/3721479428 shutdown_connections 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.958+0000 7f09e1488640 1 --2- 172.21.15.103:0/3721479428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc101400 0x7f09dc10d930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.958+0000 7f09e1488640 1 --2- 172.21.15.103:0/3721479428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc100a60 0x7f09dc100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.958+0000 7f09e1488640 1 --2- 172.21.15.103:0/3721479428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09dc106a60 0x7f09dc106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.958+0000 7f09e1488640 1 -- 172.21.15.103:0/3721479428 >> 172.21.15.103:0/3721479428 conn(0x7f09dc0fc780 msgr2=0x7f09dc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.958+0000 7f09e1488640 1 -- 172.21.15.103:0/3721479428 shutdown_connections 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.958+0000 7f09e1488640 1 -- 172.21.15.103:0/3721479428 wait complete. 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09e1488640 1 Processor -- start 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09e1488640 1 -- start start 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09e1488640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc100a60 0x7f09dc197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09e1488640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09dc101400 0x7f09dc197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09e1488640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc106a60 0x7f09dc19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09e1488640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc110d50 con 0x7f09dc100a60 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09e1488640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc110bd0 con 0x7f09dc101400 2024-09-05T02:01:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09e1488640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc110ed0 con 0x7f09dc106a60 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09daffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc100a60 0x7f09dc197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09daffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc100a60 0x7f09dc197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45228/0 (socket says 172.21.15.103:45228) 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09daffd640 1 -- 172.21.15.103:0/1178051265 learned_addr learned my addr 172.21.15.103:0/1178051265 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.959+0000 7f09db7fe640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc106a60 0x7f09dc19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09da7fc640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09dc101400 0x7f09dc197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09daffd640 1 -- 172.21.15.103:0/1178051265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc106a60 msgr2=0x7f09dc19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09daffd640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc106a60 0x7f09dc19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09daffd640 1 -- 172.21.15.103:0/1178051265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09dc101400 msgr2=0x7f09dc197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09daffd640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09dc101400 0x7f09dc197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09daffd640 1 -- 172.21.15.103:0/1178051265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc19c2c0 con 0x7f09dc100a60 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09da7fc640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09dc101400 0x7f09dc197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09daffd640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc100a60 0x7f09dc197310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f09d400df40 tx=0x7f09d400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09d3fff640 1 -- 172.21.15.103:0/1178051265 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d400ea60 con 0x7f09dc100a60 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09d3fff640 1 -- 172.21.15.103:0/1178051265 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09d400ec00 con 0x7f09dc100a60 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc19c550 con 0x7f09dc100a60 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09d3fff640 1 -- 172.21.15.103:0/1178051265 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4016670 con 0x7f09dc100a60 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.960+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc06a790 con 0x7f09dc100a60 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.961+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc1021a0 con 0x7f09dc100a60 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.962+0000 7f09d3fff640 1 -- 172.21.15.103:0/1178051265 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f09d4009580 con 0x7f09dc100a60 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.963+0000 7f09d3fff640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f09c80a2c10 0x7f09c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.963+0000 7f09da7fc640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f09c80a2c10 0x7f09c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.963+0000 7f09d3fff640 1 -- 172.21.15.103:0/1178051265 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f09d40d0d40 con 0x7f09dc100a60 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.964+0000 7f09da7fc640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f09c80a2c10 0x7f09c80a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f09c40058c0 tx=0x7f09c4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:44.966+0000 7f09d3fff640 1 -- 172.21.15.103:0/1178051265 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09d409d980 con 0x7f09dc100a60 2024-09-05T02:01:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.139+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f09dc100ec0 con 0x7f09dc100a60 2024-09-05T02:01:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.139+0000 7f09d3fff640 1 -- 172.21.15.103:0/1178051265 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f09dc100ec0 con 0x7f09dc100a60 2024-09-05T02:01:45.141 INFO:teuthology.orchestra.run.smithi103.stdout:236223201327 2024-09-05T02:01:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f09c80a2c10 msgr2=0x7f09c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f09c80a2c10 0x7f09c80a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f09c40058c0 tx=0x7f09c4005850 comp rx=0 tx=0).stop 2024-09-05T02:01:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc100a60 msgr2=0x7f09dc197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc100a60 0x7f09dc197310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f09d400df40 tx=0x7f09d400df70 comp rx=0 tx=0).stop 2024-09-05T02:01:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 shutdown_connections 2024-09-05T02:01:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f09c80a2c10 0x7f09c80a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc106a60 0x7f09dc19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09dc101400 0x7f09dc197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 --2- 172.21.15.103:0/1178051265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09dc100a60 0x7f09dc197310 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.141+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 >> 172.21.15.103:0/1178051265 conn(0x7f09dc0fc780 msgr2=0x7f09dc0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.142+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 shutdown_connections 2024-09-05T02:01:45.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:45.142+0000 7f09e1488640 1 -- 172.21.15.103:0/1178051265 wait complete. 2024-09-05T02:01:45.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:45 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1178051265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T02:01:45.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:45 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1178051265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T02:01:45.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:45 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1178051265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T02:01:45.890 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201325 got 236223201327 for osd.8 2024-09-05T02:01:45.891 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:46.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.464+0000 7f32c33a2640 1 -- 172.21.15.103:0/113398729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc102d40 msgr2=0x7f32bc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.464+0000 7f32c33a2640 1 --2- 172.21.15.103:0/113398729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc102d40 0x7f32bc103120 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009900 tx=0x7f32b802a480 comp rx=0 tx=0).stop 2024-09-05T02:01:46.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.464+0000 7f32c33a2640 1 -- 172.21.15.103:0/113398729 shutdown_connections 2024-09-05T02:01:46.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.464+0000 7f32c33a2640 1 --2- 172.21.15.103:0/113398729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc10c170 0x7f32bc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.464+0000 7f32c33a2640 1 --2- 172.21.15.103:0/113398729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32bc103660 0x7f32bc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.464+0000 7f32c33a2640 1 --2- 172.21.15.103:0/113398729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc102d40 0x7f32bc103120 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.464+0000 7f32c33a2640 1 -- 172.21.15.103:0/113398729 >> 172.21.15.103:0/113398729 conn(0x7f32bc0fc7a0 msgr2=0x7f32bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:46.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.464+0000 7f32c33a2640 1 -- 172.21.15.103:0/113398729 shutdown_connections 2024-09-05T02:01:46.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.464+0000 7f32c33a2640 1 -- 172.21.15.103:0/113398729 wait complete. 2024-09-05T02:01:46.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c33a2640 1 Processor -- start 2024-09-05T02:01:46.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c33a2640 1 -- start start 2024-09-05T02:01:46.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c33a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32bc102d40 0x7f32bc198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c33a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc103660 0x7f32bc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c33a2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc10c170 0x7f32bc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c33a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc111ee0 con 0x7f32bc103660 2024-09-05T02:01:46.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c33a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc111d60 con 0x7f32bc102d40 2024-09-05T02:01:46.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c33a2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc112060 con 0x7f32bc10c170 2024-09-05T02:01:46.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c093d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc103660 0x7f32bc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c093d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc103660 0x7f32bc198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45264/0 (socket says 172.21.15.103:45264) 2024-09-05T02:01:46.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c093d640 1 -- 172.21.15.103:0/3401781061 learned_addr learned my addr 172.21.15.103:0/3401781061 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:46.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c113e640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32bc102d40 0x7f32bc198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c193f640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc10c170 0x7f32bc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c093d640 1 -- 172.21.15.103:0/3401781061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc10c170 msgr2=0x7f32bc19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c093d640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc10c170 0x7f32bc19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c093d640 1 -- 172.21.15.103:0/3401781061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32bc102d40 msgr2=0x7f32bc198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c093d640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32bc102d40 0x7f32bc198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c093d640 1 -- 172.21.15.103:0/3401781061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32bc19d530 con 0x7f32bc103660 2024-09-05T02:01:46.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c113e640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32bc102d40 0x7f32bc198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:46.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.465+0000 7f32c093d640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc103660 0x7f32bc198ac0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b3f0 tx=0x7f32ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:46.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.466+0000 7f32b27fc640 1 -- 172.21.15.103:0/3401781061 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac015020 con 0x7f32bc103660 2024-09-05T02:01:46.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.466+0000 7f32b27fc640 1 -- 172.21.15.103:0/3401781061 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32ac011ce0 con 0x7f32bc103660 2024-09-05T02:01:46.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.466+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32bc19d820 con 0x7f32bc103660 2024-09-05T02:01:46.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.466+0000 7f32b27fc640 1 -- 172.21.15.103:0/3401781061 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac010d80 con 0x7f32bc103660 2024-09-05T02:01:46.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.466+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32bc1a5100 con 0x7f32bc103660 2024-09-05T02:01:46.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.467+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32bc1031a0 con 0x7f32bc103660 2024-09-05T02:01:46.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.468+0000 7f32b27fc640 1 -- 172.21.15.103:0/3401781061 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f32ac018020 con 0x7f32bc103660 2024-09-05T02:01:46.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.468+0000 7f32b27fc640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f32a80a2c10 0x7f32a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.469+0000 7f32c113e640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f32a80a2c10 0x7f32a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.469+0000 7f32b27fc640 1 -- 172.21.15.103:0/3401781061 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f32ac0ced20 con 0x7f32bc103660 2024-09-05T02:01:46.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.469+0000 7f32c113e640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f32a80a2c10 0x7f32a80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32b8031440 tx=0x7f32b80095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:46.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.472+0000 7f32b27fc640 1 -- 172.21.15.103:0/3401781061 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32ac01f020 con 0x7f32bc103660 2024-09-05T02:01:46.586 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:46 smithi103 ceph-mon[30902]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-05T02:01:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:46 smithi167 ceph-mon[35122]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-05T02:01:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:46 smithi139 ceph-mon[34262]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-05T02:01:46.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.648+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f32bc107210 con 0x7f32bc103660 2024-09-05T02:01:46.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.649+0000 7f32b27fc640 1 -- 172.21.15.103:0/3401781061 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f32ac09b8d0 con 0x7f32bc103660 2024-09-05T02:01:46.651 INFO:teuthology.orchestra.run.smithi103.stdout:317827579935 2024-09-05T02:01:46.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f32a80a2c10 msgr2=0x7f32a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f32a80a2c10 0x7f32a80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32b8031440 tx=0x7f32b80095f0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc103660 msgr2=0x7f32bc198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc103660 0x7f32bc198ac0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b3f0 tx=0x7f32ac00b8b0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 shutdown_connections 2024-09-05T02:01:46.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f32a80a2c10 0x7f32a80a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32bc10c170 0x7f32bc19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc103660 0x7f32bc198ac0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 --2- 172.21.15.103:0/3401781061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32bc102d40 0x7f32bc198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 >> 172.21.15.103:0/3401781061 conn(0x7f32bc0fc7a0 msgr2=0x7f32bc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:46.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 shutdown_connections 2024-09-05T02:01:46.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.651+0000 7f32c33a2640 1 -- 172.21.15.103:0/3401781061 wait complete. 2024-09-05T02:01:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.668+0000 7f2178b9e640 1 -- 172.21.15.103:0/2849521589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 msgr2=0x7f2174101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.668+0000 7f2178b9e640 1 --2- 172.21.15.103:0/2849521589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 0x7f2174101850 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2164009900 tx=0x7f216402a410 comp rx=0 tx=0).stop 2024-09-05T02:01:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.668+0000 7f2178b9e640 1 -- 172.21.15.103:0/2849521589 shutdown_connections 2024-09-05T02:01:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.668+0000 7f2178b9e640 1 --2- 172.21.15.103:0/2849521589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2174105de0 0x7f217410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.668+0000 7f2178b9e640 1 --2- 172.21.15.103:0/2849521589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2174101e20 0x7f21741056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.668+0000 7f2178b9e640 1 --2- 172.21.15.103:0/2849521589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 0x7f2174101850 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.668+0000 7f2178b9e640 1 -- 172.21.15.103:0/2849521589 >> 172.21.15.103:0/2849521589 conn(0x7f2174077230 msgr2=0x7f21740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.668+0000 7f2178b9e640 1 -- 172.21.15.103:0/2849521589 shutdown_connections 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.668+0000 7f2178b9e640 1 -- 172.21.15.103:0/2849521589 wait complete. 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.669+0000 7f2178b9e640 1 Processor -- start 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.669+0000 7f2178b9e640 1 -- start start 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.669+0000 7f2178b9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 0x7f2174197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.669+0000 7f2178b9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2174101e20 0x7f2174197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.669+0000 7f2178b9e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2174105de0 0x7f217419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.669+0000 7f2178b9e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2174110cf0 con 0x7f2174101470 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.669+0000 7f2178b9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2174110b70 con 0x7f2174101e20 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.669+0000 7f2178b9e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2174110e70 con 0x7f2174105de0 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f217259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 0x7f2174197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f217259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 0x7f2174197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45278/0 (socket says 172.21.15.103:45278) 2024-09-05T02:01:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f217259c640 1 -- 172.21.15.103:0/195919411 learned_addr learned my addr 172.21.15.103:0/195919411 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f2172d9d640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2174105de0 0x7f217419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f2171d9b640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2174101e20 0x7f2174197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f217259c640 1 -- 172.21.15.103:0/195919411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2174105de0 msgr2=0x7f217419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f217259c640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2174105de0 0x7f217419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f217259c640 1 -- 172.21.15.103:0/195919411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2174101e20 msgr2=0x7f2174197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f217259c640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2174101e20 0x7f2174197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f217259c640 1 -- 172.21.15.103:0/195919411 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217419c200 con 0x7f2174101470 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.670+0000 7f217259c640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 0x7f2174197250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2164005940 tx=0x7f2164004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.671+0000 7f21637fe640 1 -- 172.21.15.103:0/195919411 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164046070 con 0x7f2174101470 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.671+0000 7f21637fe640 1 -- 172.21.15.103:0/195919411 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2164003aa0 con 0x7f2174101470 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.671+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217419c490 con 0x7f2174101470 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.671+0000 7f21637fe640 1 -- 172.21.15.103:0/195919411 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164035050 con 0x7f2174101470 2024-09-05T02:01:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.671+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2174100100 con 0x7f2174101470 2024-09-05T02:01:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.672+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2174102220 con 0x7f2174101470 2024-09-05T02:01:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.673+0000 7f21637fe640 1 -- 172.21.15.103:0/195919411 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2164049070 con 0x7f2174101470 2024-09-05T02:01:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.674+0000 7f21637fe640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f215c0a2c90 0x7f215c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.674+0000 7f21637fe640 1 -- 172.21.15.103:0/195919411 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f21640f1230 con 0x7f2174101470 2024-09-05T02:01:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.674+0000 7f2171d9b640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f215c0a2c90 0x7f215c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.675+0000 7f2171d9b640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f215c0a2c90 0x7f215c0a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f21740ff930 tx=0x7f216800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.677+0000 7f21637fe640 1 -- 172.21.15.103:0/195919411 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f216407e1b0 con 0x7f2174101470 2024-09-05T02:01:46.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.850+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2174101850 con 0x7f2174101470 2024-09-05T02:01:46.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.851+0000 7f21637fe640 1 -- 172.21.15.103:0/195919411 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2174101850 con 0x7f2174101470 2024-09-05T02:01:46.852 INFO:teuthology.orchestra.run.smithi103.stdout:73014444109 2024-09-05T02:01:46.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f215c0a2c90 msgr2=0x7f215c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f215c0a2c90 0x7f215c0a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f21740ff930 tx=0x7f216800a2d0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 msgr2=0x7f2174197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 0x7f2174197250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2164005940 tx=0x7f2164004270 comp rx=0 tx=0).stop 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 shutdown_connections 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f215c0a2c90 0x7f215c0a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2174105de0 0x7f217419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2174101e20 0x7f2174197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 --2- 172.21.15.103:0/195919411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2174101470 0x7f2174197250 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 >> 172.21.15.103:0/195919411 conn(0x7f2174077230 msgr2=0x7f2174104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 shutdown_connections 2024-09-05T02:01:46.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.853+0000 7f2178b9e640 1 -- 172.21.15.103:0/195919411 wait complete. 2024-09-05T02:01:46.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.068+0000 7f39d3b48640 1 -- 172.21.15.103:0/1337652166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc106a80 msgr2=0x7f39cc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.068+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1337652166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc106a80 0x7f39cc106e60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009930 tx=0x7f39bc02a420 comp rx=0 tx=0).stop 2024-09-05T02:01:46.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.068+0000 7f39d3b48640 1 -- 172.21.15.103:0/1337652166 shutdown_connections 2024-09-05T02:01:46.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.068+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1337652166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39cc101420 0x7f39cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.068+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1337652166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc100a80 0x7f39cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.068+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1337652166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc106a80 0x7f39cc106e60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.068+0000 7f39d3b48640 1 -- 172.21.15.103:0/1337652166 >> 172.21.15.103:0/1337652166 conn(0x7f39cc0fc7a0 msgr2=0x7f39cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:46.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.068+0000 7f39d3b48640 1 -- 172.21.15.103:0/1337652166 shutdown_connections 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.068+0000 7f39d3b48640 1 -- 172.21.15.103:0/1337652166 wait complete. 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d3b48640 1 Processor -- start 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d3b48640 1 -- start start 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d3b48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc100a80 0x7f39cc0fced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d3b48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39cc101420 0x7f39cc0fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d3b48640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc106a80 0x7f39cc0fd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d3b48640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc110d30 con 0x7f39cc100a80 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d3b48640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc110bb0 con 0x7f39cc101420 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d3b48640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc110eb0 con 0x7f39cc106a80 2024-09-05T02:01:46.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d18e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc100a80 0x7f39cc0fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.982 INFO:teuthology.orchestra.run.smithi103.stdout:206158430259 2024-09-05T02:01:46.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d18e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc100a80 0x7f39cc0fced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45242/0 (socket says 172.21.15.103:45242) 2024-09-05T02:01:46.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d18e4640 1 -- 172.21.15.103:0/1210441040 learned_addr learned my addr 172.21.15.103:0/1210441040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:46.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d10e3640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39cc101420 0x7f39cc0fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.069+0000 7f39d20e5640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc106a80 0x7f39cc0fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39d18e4640 1 -- 172.21.15.103:0/1210441040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc106a80 msgr2=0x7f39cc0fd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39d18e4640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc106a80 0x7f39cc0fd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39d18e4640 1 -- 172.21.15.103:0/1210441040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39cc101420 msgr2=0x7f39cc0fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39d18e4640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39cc101420 0x7f39cc0fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39d18e4640 1 -- 172.21.15.103:0/1210441040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc1a68c0 con 0x7f39cc100a80 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39d10e3640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39cc101420 0x7f39cc0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39d18e4640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc100a80 0x7f39cc0fced0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f39bc005940 tx=0x7f39bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39c2ffd640 1 -- 172.21.15.103:0/1210441040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc03f030 con 0x7f39cc100a80 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39c2ffd640 1 -- 172.21.15.103:0/1210441040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39bc003cb0 con 0x7f39cc100a80 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc1a6b50 con 0x7f39cc100a80 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39c2ffd640 1 -- 172.21.15.103:0/1210441040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc035070 con 0x7f39cc100a80 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.070+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc1a7030 con 0x7f39cc100a80 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.071+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39cc100ee0 con 0x7f39cc100a80 2024-09-05T02:01:46.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.072+0000 7f39c2ffd640 1 -- 172.21.15.103:0/1210441040 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f39bc042030 con 0x7f39cc100a80 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.073+0000 7f39c2ffd640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f39b80a2c10 0x7f39b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.073+0000 7f39d10e3640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f39b80a2c10 0x7f39b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.073+0000 7f39c2ffd640 1 -- 172.21.15.103:0/1210441040 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f39bc0efdf0 con 0x7f39cc100a80 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.073+0000 7f39d10e3640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f39b80a2c10 0x7f39b80a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005fd0 tx=0x7f39c8005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.077+0000 7f39c2ffd640 1 -- 172.21.15.103:0/1210441040 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39bc0bc830 con 0x7f39cc100a80 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.249+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f39cc109a70 con 0x7f39cc100a80 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.250+0000 7f39c2ffd640 1 -- 172.21.15.103:0/1210441040 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f39bc0bc130 con 0x7f39cc100a80 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f39b80a2c10 msgr2=0x7f39b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f39b80a2c10 0x7f39b80a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005fd0 tx=0x7f39c8005ea0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc100a80 msgr2=0x7f39cc0fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc100a80 0x7f39cc0fced0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f39bc005940 tx=0x7f39bc0042f0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 shutdown_connections 2024-09-05T02:01:46.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f39b80a2c10 0x7f39b80a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc106a80 0x7f39cc0fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39cc101420 0x7f39cc0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 --2- 172.21.15.103:0/1210441040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39cc100a80 0x7f39cc0fced0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:46.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 >> 172.21.15.103:0/1210441040 conn(0x7f39cc0fc7a0 msgr2=0x7f39cc06ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:46.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 shutdown_connections 2024-09-05T02:01:46.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.252+0000 7f39d3b48640 1 -- 172.21.15.103:0/1210441040 wait complete. 2024-09-05T02:01:47.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:47 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1210441040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T02:01:47.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:47 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3401781061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T02:01:47.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:47 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/195919411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T02:01:47.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:47 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/99482819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T02:01:47.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:47 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1210441040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T02:01:47.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:47 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3401781061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T02:01:47.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:47 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/195919411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T02:01:47.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:47 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/99482819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T02:01:47.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:47 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1210441040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T02:01:47.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:47 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3401781061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T02:01:47.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:47 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/195919411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T02:01:47.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:47 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/99482819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T02:01:48.242 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579933 got 317827579935 for osd.11 2024-09-05T02:01:48.242 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:48.602 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:48 smithi103 ceph-mon[30902]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:01:48.631 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430257 got 206158430259 for osd.7 2024-09-05T02:01:48.631 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:48 smithi167 ceph-mon[35122]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:01:48.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:48 smithi139 ceph-mon[34262]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:01:49.380 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972391 got 266287972394 for osd.9 2024-09-05T02:01:49.380 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:49.779 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444106 got 73014444109 for osd.2 2024-09-05T02:01:49.779 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:49.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.902+0000 7feec2bcb640 1 -- 172.21.15.103:0/2839109714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1036e0 msgr2=0x7feebc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:49.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.902+0000 7feec2bcb640 1 --2- 172.21.15.103:0/2839109714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1036e0 0x7feebc107b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feea8009930 tx=0x7feea802a420 comp rx=0 tx=0).stop 2024-09-05T02:01:49.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 -- 172.21.15.103:0/2839109714 shutdown_connections 2024-09-05T02:01:49.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 --2- 172.21.15.103:0/2839109714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feebc1082a0 0x7feebc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 --2- 172.21.15.103:0/2839109714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1036e0 0x7feebc107b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 --2- 172.21.15.103:0/2839109714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc102d30 0x7feebc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 -- 172.21.15.103:0/2839109714 >> 172.21.15.103:0/2839109714 conn(0x7feebc0fc7d0 msgr2=0x7feebc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 -- 172.21.15.103:0/2839109714 shutdown_connections 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 -- 172.21.15.103:0/2839109714 wait complete. 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 Processor -- start 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 -- start start 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc102d30 0x7feebc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feebc1036e0 0x7feebc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1082a0 0x7feebc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feebc111f10 con 0x7feebc1082a0 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feebc111d90 con 0x7feebc102d30 2024-09-05T02:01:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.903+0000 7feec2bcb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feebc112090 con 0x7feebc1036e0 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec1168640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1082a0 0x7feebc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec1168640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1082a0 0x7feebc101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45294/0 (socket says 172.21.15.103:45294) 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec1168640 1 -- 172.21.15.103:0/99482819 learned_addr learned my addr 172.21.15.103:0/99482819 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feebbfff640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feebc1036e0 0x7feebc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec0967640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc102d30 0x7feebc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec1168640 1 -- 172.21.15.103:0/99482819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feebc1036e0 msgr2=0x7feebc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec1168640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feebc1036e0 0x7feebc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec1168640 1 -- 172.21.15.103:0/99482819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc102d30 msgr2=0x7feebc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec1168640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc102d30 0x7feebc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec1168640 1 -- 172.21.15.103:0/99482819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc1a6a60 con 0x7feebc1082a0 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec1168640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1082a0 0x7feebc101410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feeb400eaa0 tx=0x7feeb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:49.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feeb9ffb640 1 -- 172.21.15.103:0/99482819 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb400cdb0 con 0x7feebc1082a0 2024-09-05T02:01:49.882 INFO:teuthology.orchestra.run.smithi103.stdout:34359738455 2024-09-05T02:01:49.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feeb9ffb640 1 -- 172.21.15.103:0/99482819 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feeb4005020 con 0x7feebc1082a0 2024-09-05T02:01:49.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc1a6d50 con 0x7feebc1082a0 2024-09-05T02:01:49.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feeb9ffb640 1 -- 172.21.15.103:0/99482819 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40116d0 con 0x7feebc1082a0 2024-09-05T02:01:49.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.904+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc1a7210 con 0x7feebc1082a0 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.906+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc103110 con 0x7feebc1082a0 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.907+0000 7feeb9ffb640 1 -- 172.21.15.103:0/99482819 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7feeb40051d0 con 0x7feebc1082a0 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.907+0000 7feeb9ffb640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feeac0a2c10 0x7feeac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.907+0000 7feec0967640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feeac0a2c10 0x7feeac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.907+0000 7feeb9ffb640 1 -- 172.21.15.103:0/99482819 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7feeb40cf770 con 0x7feebc1082a0 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.908+0000 7feec0967640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feeac0a2c10 0x7feeac0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feeb00058c0 tx=0x7feeb000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:46.911+0000 7feeb9ffb640 1 -- 172.21.15.103:0/99482819 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feeb4016030 con 0x7feebc1082a0 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.081+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7feebc10a2b0 con 0x7feebc1082a0 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.082+0000 7feeb9ffb640 1 -- 172.21.15.103:0/99482819 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feeb409c210 con 0x7feebc1082a0 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feeac0a2c10 msgr2=0x7feeac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feeac0a2c10 0x7feeac0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feeb00058c0 tx=0x7feeb000a4c0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1082a0 msgr2=0x7feebc101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:49.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1082a0 0x7feebc101410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feeb400eaa0 tx=0x7feeb400ef60 comp rx=0 tx=0).stop 2024-09-05T02:01:49.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 shutdown_connections 2024-09-05T02:01:49.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feebc1082a0 0x7feebc101410 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feeac0a2c10 0x7feeac0a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feebc1036e0 0x7feebc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 --2- 172.21.15.103:0/99482819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc102d30 0x7feebc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:49.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 >> 172.21.15.103:0/99482819 conn(0x7feebc0fc7d0 msgr2=0x7feebc105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:49.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 shutdown_connections 2024-09-05T02:01:49.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:47.084+0000 7feec2bcb640 1 -- 172.21.15.103:0/99482819 wait complete. 2024-09-05T02:01:50.543 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:50 smithi103 ceph-mon[30902]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:01:50.578 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738455 for osd.0 2024-09-05T02:01:50.578 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:50 smithi167 ceph-mon[35122]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:01:50.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:50 smithi139 ceph-mon[34262]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:01:51.274 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808868 for osd.10 2024-09-05T02:01:51.275 DEBUG:teuthology.parallel:result is None 2024-09-05T02:01:51.275 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T02:01:51.275 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T02:01:51.527 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 -- 172.21.15.103:0/2820600097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c101df0 msgr2=0x7fab7c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 --2- 172.21.15.103:0/2820600097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c101df0 0x7fab7c1021d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fab78009900 tx=0x7fab7802a410 comp rx=0 tx=0).stop 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 -- 172.21.15.103:0/2820600097 shutdown_connections 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 --2- 172.21.15.103:0/2820600097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab7c106f20 0x7fab7c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 --2- 172.21.15.103:0/2820600097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab7c1027a0 0x7fab7c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 --2- 172.21.15.103:0/2820600097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c101df0 0x7fab7c1021d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 -- 172.21.15.103:0/2820600097 >> 172.21.15.103:0/2820600097 conn(0x7fab7c0fd8f0 msgr2=0x7fab7c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 -- 172.21.15.103:0/2820600097 shutdown_connections 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 -- 172.21.15.103:0/2820600097 wait complete. 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 Processor -- start 2024-09-05T02:01:52.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 -- start start 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab7c101df0 0x7fab7c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.022+0000 7fab839fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab7c1027a0 0x7fab7c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab839fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c106f20 0x7fab7c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab839fe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c111e50 con 0x7fab7c106f20 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab839fe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c111cd0 con 0x7fab7c101df0 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab839fe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c111fd0 con 0x7fab7c1027a0 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab81f9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c106f20 0x7fab7c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab81f9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c106f20 0x7fab7c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60726/0 (socket says 172.21.15.103:60726) 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab81f9b640 1 -- 172.21.15.103:0/1704706604 learned_addr learned my addr 172.21.15.103:0/1704706604 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab8179a640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab7c101df0 0x7fab7c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab80f99640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab7c1027a0 0x7fab7c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab81f9b640 1 -- 172.21.15.103:0/1704706604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab7c1027a0 msgr2=0x7fab7c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab81f9b640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab7c1027a0 0x7fab7c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab81f9b640 1 -- 172.21.15.103:0/1704706604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab7c101df0 msgr2=0x7fab7c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab81f9b640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab7c101df0 0x7fab7c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab81f9b640 1 -- 172.21.15.103:0/1704706604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c19d400 con 0x7fab7c106f20 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab8179a640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab7c101df0 0x7fab7c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab80f99640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab7c1027a0 0x7fab7c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab81f9b640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c106f20 0x7fab7c19cd20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fab7400cce0 tx=0x7fab7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab727fc640 1 -- 172.21.15.103:0/1704706604 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7400f450 con 0x7fab7c106f20 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab727fc640 1 -- 172.21.15.103:0/1704706604 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab7400faa0 con 0x7fab7c106f20 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c19d6f0 con 0x7fab7c106f20 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab727fc640 1 -- 172.21.15.103:0/1704706604 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74018b70 con 0x7fab7c106f20 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.023+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1a4fd0 con 0x7fab7c106f20 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.024+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7c1021d0 con 0x7fab7c106f20 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.025+0000 7fab727fc640 1 -- 172.21.15.103:0/1704706604 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fab74018d10 con 0x7fab7c106f20 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.026+0000 7fab727fc640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab680a2c10 0x7fab680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:52.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.026+0000 7fab8179a640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab680a2c10 0x7fab680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:52.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.026+0000 7fab727fc640 1 -- 172.21.15.103:0/1704706604 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fab740cfa70 con 0x7fab7c106f20 2024-09-05T02:01:52.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.027+0000 7fab8179a640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab680a2c10 0x7fab680a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab78031440 tx=0x7fab780095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:52.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.030+0000 7fab727fc640 1 -- 172.21.15.103:0/1704706604 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab74014030 con 0x7fab7c106f20 2024-09-05T02:01:52.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.195+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab7c10a160 con 0x7fab680a2c10 2024-09-05T02:01:52.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.203+0000 7fab727fc640 1 -- 172.21.15.103:0/1704706604 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340395 (secure 0 0 0) 0x7fab7c10a160 con 0x7fab680a2c10 2024-09-05T02:01:52.208 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:52.210 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-05T02:01:52.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab680a2c10 msgr2=0x7fab680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:52.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab680a2c10 0x7fab680a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab78031440 tx=0x7fab780095f0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c106f20 msgr2=0x7fab7c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:52.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c106f20 0x7fab7c19cd20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fab7400cce0 tx=0x7fab7400a620 comp rx=0 tx=0).stop 2024-09-05T02:01:52.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 shutdown_connections 2024-09-05T02:01:52.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab7c106f20 0x7fab7c19cd20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fab680a2c10 0x7fab680a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab7c1027a0 0x7fab7c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 --2- 172.21.15.103:0/1704706604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab7c101df0 0x7fab7c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:52.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 >> 172.21.15.103:0/1704706604 conn(0x7fab7c0fd8f0 msgr2=0x7fab7c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:52.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 shutdown_connections 2024-09-05T02:01:52.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:52.207+0000 7fab839fe640 1 -- 172.21.15.103:0/1704706604 wait complete. 2024-09-05T02:01:52.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:52 smithi103 ceph-mon[30902]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:01:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:52 smithi167 ceph-mon[35122]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:01:52.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:52 smithi139 ceph-mon[34262]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:01:52.906 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-05T02:01:51.105668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659886,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":666,"num_read_kb":455,"num_write":432,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392384,"kb_used_data":13940,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549440,"statfs":{"total":1151940427776,"available":1151538626560,"internally_reserved":0,"allocated":14274560,"data_stored":7915484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003055"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:47.305002+0000","last_change":"2024-09-05T02:00:12.347638+0000","last_active":"2024-09-05T02:01:47.305002+0000","last_peered":"2024-09-05T02:01:47.305002+0000","last_clean":"2024-09-05T02:01:47.305002+0000","last_became_active":"2024-09-05T02:00:12.347314+0000","last_became_peered":"2024-09-05T02:00:12.347314+0000","last_unstale":"2024-09-05T02:01:47.305002+0000","last_undegraded":"2024-09-05T02:01:47.305002+0000","last_fullsized":"2024-09-05T02:01:47.305002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:08:39.084117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507253+0000","last_change":"2024-09-05T02:00:14.360262+0000","last_active":"2024-09-05T02:01:26.507253+0000","last_peered":"2024-09-05T02:01:26.507253+0000","last_clean":"2024-09-05T02:01:26.507253+0000","last_became_active":"2024-09-05T02:00:14.359965+0000","last_became_peered":"2024-09-05T02:00:14.359965+0000","last_unstale":"2024-09-05T02:01:26.507253+0000","last_undegraded":"2024-09-05T02:01:26.507253+0000","last_fullsized":"2024-09-05T02:01:26.507253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:01:56.109169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265425+0000","last_change":"2024-09-05T02:00:21.410692+0000","last_active":"2024-09-05T02:01:26.265425+0000","last_peered":"2024-09-05T02:01:26.265425+0000","last_clean":"2024-09-05T02:01:26.265425+0000","last_became_active":"2024-09-05T02:00:21.410298+0000","last_became_peered":"2024-09-05T02:00:21.410298+0000","last_unstale":"2024-09-05T02:01:26.265425+0000","last_undegraded":"2024-09-05T02:01:26.265425+0000","last_fullsized":"2024-09-05T02:01:26.265425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:25:31.241700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507935+0000","last_change":"2024-09-05T02:00:12.783109+0000","last_active":"2024-09-05T02:01:26.507935+0000","last_peered":"2024-09-05T02:01:26.507935+0000","last_clean":"2024-09-05T02:01:26.507935+0000","last_became_active":"2024-09-05T02:00:12.782773+0000","last_became_peered":"2024-09-05T02:00:12.782773+0000","last_unstale":"2024-09-05T02:01:26.507935+0000","last_undegraded":"2024-09-05T02:01:26.507935+0000","last_fullsized":"2024-09-05T02:01:26.507935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:29:12.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267769+0000","last_change":"2024-09-05T02:00:14.370425+0000","last_active":"2024-09-05T02:01:26.267769+0000","last_peered":"2024-09-05T02:01:26.267769+0000","last_clean":"2024-09-05T02:01:26.267769+0000","last_became_active":"2024-09-05T02:00:14.370102+0000","last_became_peered":"2024-09-05T02:00:14.370102+0000","last_unstale":"2024-09-05T02:01:26.267769+0000","last_undegraded":"2024-09-05T02:01:26.267769+0000","last_fullsized":"2024-09-05T02:01:26.267769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:22:43.837418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268987+0000","last_change":"2024-09-05T02:00:21.413082+0000","last_active":"2024-09-05T02:01:26.268987+0000","last_peered":"2024-09-05T02:01:26.268987+0000","last_clean":"2024-09-05T02:01:26.268987+0000","last_became_active":"2024-09-05T02:00:21.412661+0000","last_became_peered":"2024-09-05T02:00:21.412661+0000","last_unstale":"2024-09-05T02:01:26.268987+0000","last_undegraded":"2024-09-05T02:01:26.268987+0000","last_fullsized":"2024-09-05T02:01:26.268987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:51:41.225573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269774+0000","last_change":"2024-09-05T02:00:12.353894+0000","last_active":"2024-09-05T02:01:26.269774+0000","last_peered":"2024-09-05T02:01:26.269774+0000","last_clean":"2024-09-05T02:01:26.269774+0000","last_became_active":"2024-09-05T02:00:12.353483+0000","last_became_peered":"2024-09-05T02:00:12.353483+0000","last_unstale":"2024-09-05T02:01:26.269774+0000","last_undegraded":"2024-09-05T02:01:26.269774+0000","last_fullsized":"2024-09-05T02:01:26.269774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:42.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507738+0000","last_change":"2024-09-05T02:00:14.356441+0000","last_active":"2024-09-05T02:01:26.507738+0000","last_peered":"2024-09-05T02:01:26.507738+0000","last_clean":"2024-09-05T02:01:26.507738+0000","last_became_active":"2024-09-05T02:00:14.356092+0000","last_became_peered":"2024-09-05T02:00:14.356092+0000","last_unstale":"2024-09-05T02:01:26.507738+0000","last_undegraded":"2024-09-05T02:01:26.507738+0000","last_fullsized":"2024-09-05T02:01:26.507738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:33:55.416968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269847+0000","last_change":"2024-09-05T02:00:21.418324+0000","last_active":"2024-09-05T02:01:26.269847+0000","last_peered":"2024-09-05T02:01:26.269847+0000","last_clean":"2024-09-05T02:01:26.269847+0000","last_became_active":"2024-09-05T02:00:21.417934+0000","last_became_peered":"2024-09-05T02:00:21.417934+0000","last_unstale":"2024-09-05T02:01:26.269847+0000","last_undegraded":"2024-09-05T02:01:26.269847+0000","last_fullsized":"2024-09-05T02:01:26.269847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:50.425823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269482+0000","last_change":"2024-09-05T02:00:12.348637+0000","last_active":"2024-09-05T02:01:26.269482+0000","last_peered":"2024-09-05T02:01:26.269482+0000","last_clean":"2024-09-05T02:01:26.269482+0000","last_became_active":"2024-09-05T02:00:12.348287+0000","last_became_peered":"2024-09-05T02:00:12.348287+0000","last_unstale":"2024-09-05T02:01:26.269482+0000","last_undegraded":"2024-09-05T02:01:26.269482+0000","last_fullsized":"2024-09-05T02:01:26.269482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:52:58.368026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268527+0000","last_change":"2024-09-05T02:00:14.368414+0000","last_active":"2024-09-05T02:01:26.268527+0000","last_peered":"2024-09-05T02:01:26.268527+0000","last_clean":"2024-09-05T02:01:26.268527+0000","last_became_active":"2024-09-05T02:00:14.368018+0000","last_became_peered":"2024-09-05T02:00:14.368018+0000","last_unstale":"2024-09-05T02:01:26.268527+0000","last_undegraded":"2024-09-05T02:01:26.268527+0000","last_fullsized":"2024-09-05T02:01:26.268527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:08.246420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268052+0000","last_change":"2024-09-05T02:00:21.416649+0000","last_active":"2024-09-05T02:01:26.268052+0000","last_peered":"2024-09-05T02:01:26.268052+0000","last_clean":"2024-09-05T02:01:26.268052+0000","last_became_active":"2024-09-05T02:00:21.416247+0000","last_became_peered":"2024-09-05T02:00:21.416247+0000","last_unstale":"2024-09-05T02:01:26.268052+0000","last_undegraded":"2024-09-05T02:01:26.268052+0000","last_fullsized":"2024-09-05T02:01:26.268052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:10:00.115655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267664+0000","last_change":"2024-09-05T02:00:14.357719+0000","last_active":"2024-09-05T02:01:26.267664+0000","last_peered":"2024-09-05T02:01:26.267664+0000","last_clean":"2024-09-05T02:01:26.267664+0000","last_became_active":"2024-09-05T02:00:14.357211+0000","last_became_peered":"2024-09-05T02:00:14.357211+0000","last_unstale":"2024-09-05T02:01:26.267664+0000","last_undegraded":"2024-09-05T02:01:26.267664+0000","last_fullsized":"2024-09-05T02:01:26.267664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:27:15.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269584+0000","last_change":"2024-09-05T02:00:12.353893+0000","last_active":"2024-09-05T02:01:26.269584+0000","last_peered":"2024-09-05T02:01:26.269584+0000","last_clean":"2024-09-05T02:01:26.269584+0000","last_became_active":"2024-09-05T02:00:12.353531+0000","last_became_peered":"2024-09-05T02:00:12.353531+0000","last_unstale":"2024-09-05T02:01:26.269584+0000","last_undegraded":"2024-09-05T02:01:26.269584+0000","last_fullsized":"2024-09-05T02:01:26.269584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:58.597446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268248+0000","last_change":"2024-09-05T02:00:21.415203+0000","last_active":"2024-09-05T02:01:26.268248+0000","last_peered":"2024-09-05T02:01:26.268248+0000","last_clean":"2024-09-05T02:01:26.268248+0000","last_became_active":"2024-09-05T02:00:21.414767+0000","last_became_peered":"2024-09-05T02:00:21.414767+0000","last_unstale":"2024-09-05T02:01:26.268248+0000","last_undegraded":"2024-09-05T02:01:26.268248+0000","last_fullsized":"2024-09-05T02:01:26.268248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:01:48.398448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.506997+0000","last_change":"2024-09-05T02:00:14.369417+0000","last_active":"2024-09-05T02:01:26.506997+0000","last_peered":"2024-09-05T02:01:26.506997+0000","last_clean":"2024-09-05T02:01:26.506997+0000","last_became_active":"2024-09-05T02:00:14.369093+0000","last_became_peered":"2024-09-05T02:00:14.369093+0000","last_unstale":"2024-09-05T02:01:26.506997+0000","last_undegraded":"2024-09-05T02:01:26.506997+0000","last_fullsized":"2024-09-05T02:01:26.506997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:01:34.239300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.266047+0000","last_change":"2024-09-05T02:00:12.347945+0000","last_active":"2024-09-05T02:01:26.266047+0000","last_peered":"2024-09-05T02:01:26.266047+0000","last_clean":"2024-09-05T02:01:26.266047+0000","last_became_active":"2024-09-05T02:00:12.347577+0000","last_became_peered":"2024-09-05T02:00:12.347577+0000","last_unstale":"2024-09-05T02:01:26.266047+0000","last_undegraded":"2024-09-05T02:01:26.266047+0000","last_fullsized":"2024-09-05T02:01:26.266047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:53:23.947159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267513+0000","last_change":"2024-09-05T02:00:21.419070+0000","last_active":"2024-09-05T02:01:26.267513+0000","last_peered":"2024-09-05T02:01:26.267513+0000","last_clean":"2024-09-05T02:01:26.267513+0000","last_became_active":"2024-09-05T02:00:21.418315+0000","last_became_peered":"2024-09-05T02:00:21.418315+0000","last_unstale":"2024-09-05T02:01:26.267513+0000","last_undegraded":"2024-09-05T02:01:26.267513+0000","last_fullsized":"2024-09-05T02:01:26.267513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:17.974047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507591+0000","last_change":"2024-09-05T02:00:14.362965+0000","last_active":"2024-09-05T02:01:26.507591+0000","last_peered":"2024-09-05T02:01:26.507591+0000","last_clean":"2024-09-05T02:01:26.507591+0000","last_became_active":"2024-09-05T02:00:14.361842+0000","last_became_peered":"2024-09-05T02:00:14.361842+0000","last_unstale":"2024-09-05T02:01:26.507591+0000","last_undegraded":"2024-09-05T02:01:26.507591+0000","last_fullsized":"2024-09-05T02:01:26.507591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:45:10.565392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507726+0000","last_change":"2024-09-05T02:00:12.349368+0000","last_active":"2024-09-05T02:01:26.507726+0000","last_peered":"2024-09-05T02:01:26.507726+0000","last_clean":"2024-09-05T02:01:26.507726+0000","last_became_active":"2024-09-05T02:00:12.348990+0000","last_became_peered":"2024-09-05T02:00:12.348990+0000","last_unstale":"2024-09-05T02:01:26.507726+0000","last_undegraded":"2024-09-05T02:01:26.507726+0000","last_fullsized":"2024-09-05T02:01:26.507726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:45:04.998316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265703+0000","last_change":"2024-09-05T02:00:21.415467+0000","last_active":"2024-09-05T02:01:26.265703+0000","last_peered":"2024-09-05T02:01:26.265703+0000","last_clean":"2024-09-05T02:01:26.265703+0000","last_became_active":"2024-09-05T02:00:21.415138+0000","last_became_peered":"2024-09-05T02:00:21.415138+0000","last_unstale":"2024-09-05T02:01:26.265703+0000","last_undegraded":"2024-09-05T02:01:26.265703+0000","last_fullsized":"2024-09-05T02:01:26.265703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:51.221383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268684+0000","last_change":"2024-09-05T02:00:14.357961+0000","last_active":"2024-09-05T02:01:26.268684+0000","last_peered":"2024-09-05T02:01:26.268684+0000","last_clean":"2024-09-05T02:01:26.268684+0000","last_became_active":"2024-09-05T02:00:14.357624+0000","last_became_peered":"2024-09-05T02:00:14.357624+0000","last_unstale":"2024-09-05T02:01:26.268684+0000","last_undegraded":"2024-09-05T02:01:26.268684+0000","last_fullsized":"2024-09-05T02:01:26.268684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:01.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508317+0000","last_change":"2024-09-05T02:00:12.783304+0000","last_active":"2024-09-05T02:01:26.508317+0000","last_peered":"2024-09-05T02:01:26.508317+0000","last_clean":"2024-09-05T02:01:26.508317+0000","last_became_active":"2024-09-05T02:00:12.782953+0000","last_became_peered":"2024-09-05T02:00:12.782953+0000","last_unstale":"2024-09-05T02:01:26.508317+0000","last_undegraded":"2024-09-05T02:01:26.508317+0000","last_fullsized":"2024-09-05T02:01:26.508317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:50:59.130705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267510+0000","last_change":"2024-09-05T02:00:21.418357+0000","last_active":"2024-09-05T02:01:26.267510+0000","last_peered":"2024-09-05T02:01:26.267510+0000","last_clean":"2024-09-05T02:01:26.267510+0000","last_became_active":"2024-09-05T02:00:21.417779+0000","last_became_peered":"2024-09-05T02:00:21.417779+0000","last_unstale":"2024-09-05T02:01:26.267510+0000","last_undegraded":"2024-09-05T02:01:26.267510+0000","last_fullsized":"2024-09-05T02:01:26.267510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:37:45.423447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267795+0000","last_change":"2024-09-05T02:00:14.362857+0000","last_active":"2024-09-05T02:01:26.267795+0000","last_peered":"2024-09-05T02:01:26.267795+0000","last_clean":"2024-09-05T02:01:26.267795+0000","last_became_active":"2024-09-05T02:00:14.362556+0000","last_became_peered":"2024-09-05T02:00:14.362556+0000","last_unstale":"2024-09-05T02:01:26.267795+0000","last_undegraded":"2024-09-05T02:01:26.267795+0000","last_fullsized":"2024-09-05T02:01:26.267795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:44:03.413055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268748+0000","last_change":"2024-09-05T02:00:12.512747+0000","last_active":"2024-09-05T02:01:26.268748+0000","last_peered":"2024-09-05T02:01:26.268748+0000","last_clean":"2024-09-05T02:01:26.268748+0000","last_became_active":"2024-09-05T02:00:12.512471+0000","last_became_peered":"2024-09-05T02:00:12.512471+0000","last_unstale":"2024-09-05T02:01:26.268748+0000","last_undegraded":"2024-09-05T02:01:26.268748+0000","last_fullsized":"2024-09-05T02:01:26.268748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:50.235020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265537+0000","last_change":"2024-09-05T02:00:21.413947+0000","last_active":"2024-09-05T02:01:26.265537+0000","last_peered":"2024-09-05T02:01:26.265537+0000","last_clean":"2024-09-05T02:01:26.265537+0000","last_became_active":"2024-09-05T02:00:21.413488+0000","last_became_peered":"2024-09-05T02:00:21.413488+0000","last_unstale":"2024-09-05T02:01:26.265537+0000","last_undegraded":"2024-09-05T02:01:26.265537+0000","last_fullsized":"2024-09-05T02:01:26.265537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:52:16.201325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507067+0000","last_change":"2024-09-05T02:00:14.367009+0000","last_active":"2024-09-05T02:01:26.507067+0000","last_peered":"2024-09-05T02:01:26.507067+0000","last_clean":"2024-09-05T02:01:26.507067+0000","last_became_active":"2024-09-05T02:00:14.366698+0000","last_became_peered":"2024-09-05T02:00:14.366698+0000","last_unstale":"2024-09-05T02:01:26.507067+0000","last_undegraded":"2024-09-05T02:01:26.507067+0000","last_fullsized":"2024-09-05T02:01:26.507067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:37:43.811300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'16","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265846+0000","last_change":"2024-09-05T02:00:12.346918+0000","last_active":"2024-09-05T02:01:26.265846+0000","last_peered":"2024-09-05T02:01:26.265846+0000","last_clean":"2024-09-05T02:01:26.265846+0000","last_became_active":"2024-09-05T02:00:12.346541+0000","last_became_peered":"2024-09-05T02:00:12.346541+0000","last_unstale":"2024-09-05T02:01:26.265846+0000","last_undegraded":"2024-09-05T02:01:26.265846+0000","last_fullsized":"2024-09-05T02:01:26.265846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:50.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507496+0000","last_change":"2024-09-05T02:00:21.413077+0000","last_active":"2024-09-05T02:01:26.507496+0000","last_peered":"2024-09-05T02:01:26.507496+0000","last_clean":"2024-09-05T02:01:26.507496+0000","last_became_active":"2024-09-05T02:00:21.412698+0000","last_became_peered":"2024-09-05T02:00:21.412698+0000","last_unstale":"2024-09-05T02:01:26.507496+0000","last_undegraded":"2024-09-05T02:01:26.507496+0000","last_fullsized":"2024-09-05T02:01:26.507496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:12:18.463614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281275+0000","last_change":"2024-09-05T02:00:14.363072+0000","last_active":"2024-09-05T02:01:26.281275+0000","last_peered":"2024-09-05T02:01:26.281275+0000","last_clean":"2024-09-05T02:01:26.281275+0000","last_became_active":"2024-09-05T02:00:14.361932+0000","last_became_peered":"2024-09-05T02:00:14.361932+0000","last_unstale":"2024-09-05T02:01:26.281275+0000","last_undegraded":"2024-09-05T02:01:26.281275+0000","last_fullsized":"2024-09-05T02:01:26.281275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:20:13.080963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269895+0000","last_change":"2024-09-05T02:00:12.783825+0000","last_active":"2024-09-05T02:01:26.269895+0000","last_peered":"2024-09-05T02:01:26.269895+0000","last_clean":"2024-09-05T02:01:26.269895+0000","last_became_active":"2024-09-05T02:00:12.783298+0000","last_became_peered":"2024-09-05T02:00:12.783298+0000","last_unstale":"2024-09-05T02:01:26.269895+0000","last_undegraded":"2024-09-05T02:01:26.269895+0000","last_fullsized":"2024-09-05T02:01:26.269895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:00:10.986318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508069+0000","last_change":"2024-09-05T02:00:21.416474+0000","last_active":"2024-09-05T02:01:26.508069+0000","last_peered":"2024-09-05T02:01:26.508069+0000","last_clean":"2024-09-05T02:01:26.508069+0000","last_became_active":"2024-09-05T02:00:21.416006+0000","last_became_peered":"2024-09-05T02:00:21.416006+0000","last_unstale":"2024-09-05T02:01:26.508069+0000","last_undegraded":"2024-09-05T02:01:26.508069+0000","last_fullsized":"2024-09-05T02:01:26.508069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:00:18.587895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265953+0000","last_change":"2024-09-05T02:00:14.362458+0000","last_active":"2024-09-05T02:01:26.265953+0000","last_peered":"2024-09-05T02:01:26.265953+0000","last_clean":"2024-09-05T02:01:26.265953+0000","last_became_active":"2024-09-05T02:00:14.362080+0000","last_became_peered":"2024-09-05T02:00:14.362080+0000","last_unstale":"2024-09-05T02:01:26.265953+0000","last_undegraded":"2024-09-05T02:01:26.265953+0000","last_fullsized":"2024-09-05T02:01:26.265953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:47:49.498468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265949+0000","last_change":"2024-09-05T02:00:12.783445+0000","last_active":"2024-09-05T02:01:26.265949+0000","last_peered":"2024-09-05T02:01:26.265949+0000","last_clean":"2024-09-05T02:01:26.265949+0000","last_became_active":"2024-09-05T02:00:12.783131+0000","last_became_peered":"2024-09-05T02:00:12.783131+0000","last_unstale":"2024-09-05T02:01:26.265949+0000","last_undegraded":"2024-09-05T02:01:26.265949+0000","last_fullsized":"2024-09-05T02:01:26.265949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:21:57.743629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268951+0000","last_change":"2024-09-05T02:00:21.418568+0000","last_active":"2024-09-05T02:01:26.268951+0000","last_peered":"2024-09-05T02:01:26.268951+0000","last_clean":"2024-09-05T02:01:26.268951+0000","last_became_active":"2024-09-05T02:00:21.418152+0000","last_became_peered":"2024-09-05T02:00:21.418152+0000","last_unstale":"2024-09-05T02:01:26.268951+0000","last_undegraded":"2024-09-05T02:01:26.268951+0000","last_fullsized":"2024-09-05T02:01:26.268951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:50:17.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265914+0000","last_change":"2024-09-05T02:00:14.364556+0000","last_active":"2024-09-05T02:01:26.265914+0000","last_peered":"2024-09-05T02:01:26.265914+0000","last_clean":"2024-09-05T02:01:26.265914+0000","last_became_active":"2024-09-05T02:00:14.364285+0000","last_became_peered":"2024-09-05T02:00:14.364285+0000","last_unstale":"2024-09-05T02:01:26.265914+0000","last_undegraded":"2024-09-05T02:01:26.265914+0000","last_fullsized":"2024-09-05T02:01:26.265914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:24:54.901510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:45.303341+0000","last_change":"2024-09-05T02:00:12.347993+0000","last_active":"2024-09-05T02:01:45.303341+0000","last_peered":"2024-09-05T02:01:45.303341+0000","last_clean":"2024-09-05T02:01:45.303341+0000","last_became_active":"2024-09-05T02:00:12.347656+0000","last_became_peered":"2024-09-05T02:00:12.347656+0000","last_unstale":"2024-09-05T02:01:45.303341+0000","last_undegraded":"2024-09-05T02:01:45.303341+0000","last_fullsized":"2024-09-05T02:01:45.303341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:31:23.820379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281122+0000","last_change":"2024-09-05T02:00:21.410786+0000","last_active":"2024-09-05T02:01:26.281122+0000","last_peered":"2024-09-05T02:01:26.281122+0000","last_clean":"2024-09-05T02:01:26.281122+0000","last_became_active":"2024-09-05T02:00:21.410440+0000","last_became_peered":"2024-09-05T02:00:21.410440+0000","last_unstale":"2024-09-05T02:01:26.281122+0000","last_undegraded":"2024-09-05T02:01:26.281122+0000","last_fullsized":"2024-09-05T02:01:26.281122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:22:14.644413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.282147+0000","last_change":"2024-09-05T02:01:26.282147+0000","last_active":"2024-09-05T02:01:26.282147+0000","last_peered":"2024-09-05T02:01:26.282147+0000","last_clean":"2024-09-05T02:01:26.282147+0000","last_became_active":"2024-09-05T02:01:26.281792+0000","last_became_peered":"2024-09-05T02:01:26.281792+0000","last_unstale":"2024-09-05T02:01:26.282147+0000","last_undegraded":"2024-09-05T02:01:26.282147+0000","last_fullsized":"2024-09-05T02:01:26.282147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:10:43.403518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:46.303986+0000","last_change":"2024-09-05T02:00:12.783358+0000","last_active":"2024-09-05T02:01:46.303986+0000","last_peered":"2024-09-05T02:01:46.303986+0000","last_clean":"2024-09-05T02:01:46.303986+0000","last_became_active":"2024-09-05T02:00:12.782950+0000","last_became_peered":"2024-09-05T02:00:12.782950+0000","last_unstale":"2024-09-05T02:01:46.303986+0000","last_undegraded":"2024-09-05T02:01:46.303986+0000","last_fullsized":"2024-09-05T02:01:46.303986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:17:38.823019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281218+0000","last_change":"2024-09-05T02:00:21.416389+0000","last_active":"2024-09-05T02:01:26.281218+0000","last_peered":"2024-09-05T02:01:26.281218+0000","last_clean":"2024-09-05T02:01:26.281218+0000","last_became_active":"2024-09-05T02:00:21.416065+0000","last_became_peered":"2024-09-05T02:00:21.416065+0000","last_unstale":"2024-09-05T02:01:26.281218+0000","last_undegraded":"2024-09-05T02:01:26.281218+0000","last_fullsized":"2024-09-05T02:01:26.281218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:15.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.280762+0000","last_change":"2024-09-05T02:00:14.361404+0000","last_active":"2024-09-05T02:01:26.280762+0000","last_peered":"2024-09-05T02:01:26.280762+0000","last_clean":"2024-09-05T02:01:26.280762+0000","last_became_active":"2024-09-05T02:00:14.361031+0000","last_became_peered":"2024-09-05T02:00:14.361031+0000","last_unstale":"2024-09-05T02:01:26.280762+0000","last_undegraded":"2024-09-05T02:01:26.280762+0000","last_fullsized":"2024-09-05T02:01:26.280762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:40.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267876+0000","last_change":"2024-09-05T02:00:12.351656+0000","last_active":"2024-09-05T02:01:26.267876+0000","last_peered":"2024-09-05T02:01:26.267876+0000","last_clean":"2024-09-05T02:01:26.267876+0000","last_became_active":"2024-09-05T02:00:12.351310+0000","last_became_peered":"2024-09-05T02:00:12.351310+0000","last_unstale":"2024-09-05T02:01:26.267876+0000","last_undegraded":"2024-09-05T02:01:26.267876+0000","last_fullsized":"2024-09-05T02:01:26.267876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:55:27.949471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.510839+0000","last_change":"2024-09-05T02:01:26.510839+0000","last_active":"2024-09-05T02:01:26.510839+0000","last_peered":"2024-09-05T02:01:26.510839+0000","last_clean":"2024-09-05T02:01:26.510839+0000","last_became_active":"2024-09-05T02:01:26.509959+0000","last_became_peered":"2024-09-05T02:01:26.509959+0000","last_unstale":"2024-09-05T02:01:26.510839+0000","last_undegraded":"2024-09-05T02:01:26.510839+0000","last_fullsized":"2024-09-05T02:01:26.510839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:57.240190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507027+0000","last_change":"2024-09-05T02:00:14.359356+0000","last_active":"2024-09-05T02:01:26.507027+0000","last_peered":"2024-09-05T02:01:26.507027+0000","last_clean":"2024-09-05T02:01:26.507027+0000","last_became_active":"2024-09-05T02:00:14.359078+0000","last_became_peered":"2024-09-05T02:00:14.359078+0000","last_unstale":"2024-09-05T02:01:26.507027+0000","last_undegraded":"2024-09-05T02:01:26.507027+0000","last_fullsized":"2024-09-05T02:01:26.507027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:40:24.219962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.270304+0000","last_change":"2024-09-05T02:00:12.349749+0000","last_active":"2024-09-05T02:01:26.270304+0000","last_peered":"2024-09-05T02:01:26.270304+0000","last_clean":"2024-09-05T02:01:26.270304+0000","last_became_active":"2024-09-05T02:00:12.349247+0000","last_became_peered":"2024-09-05T02:00:12.349247+0000","last_unstale":"2024-09-05T02:01:26.270304+0000","last_undegraded":"2024-09-05T02:01:26.270304+0000","last_fullsized":"2024-09-05T02:01:26.270304+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:04:26.361209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268031+0000","last_change":"2024-09-05T01:58:57.552989+0000","last_active":"2024-09-05T02:01:26.268031+0000","last_peered":"2024-09-05T02:01:26.268031+0000","last_clean":"2024-09-05T02:01:26.268031+0000","last_became_active":"2024-09-05T01:58:57.552511+0000","last_became_peered":"2024-09-05T01:58:57.552511+0000","last_unstale":"2024-09-05T02:01:26.268031+0000","last_undegraded":"2024-09-05T02:01:26.268031+0000","last_fullsized":"2024-09-05T02:01:26.268031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_clean_scrub_stamp":"2024-09-05T01:55:35.678445+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:53.711210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268473+0000","last_change":"2024-09-05T02:00:21.421456+0000","last_active":"2024-09-05T02:01:26.268473+0000","last_peered":"2024-09-05T02:01:26.268473+0000","last_clean":"2024-09-05T02:01:26.268473+0000","last_became_active":"2024-09-05T02:00:21.421074+0000","last_became_peered":"2024-09-05T02:00:21.421074+0000","last_unstale":"2024-09-05T02:01:26.268473+0000","last_undegraded":"2024-09-05T02:01:26.268473+0000","last_fullsized":"2024-09-05T02:01:26.268473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:46:00.660079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268706+0000","last_change":"2024-09-05T02:00:14.357638+0000","last_active":"2024-09-05T02:01:26.268706+0000","last_peered":"2024-09-05T02:01:26.268706+0000","last_clean":"2024-09-05T02:01:26.268706+0000","last_became_active":"2024-09-05T02:00:14.357125+0000","last_became_peered":"2024-09-05T02:00:14.357125+0000","last_unstale":"2024-09-05T02:01:26.268706+0000","last_undegraded":"2024-09-05T02:01:26.268706+0000","last_fullsized":"2024-09-05T02:01:26.268706+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:02:04.343235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:48.307918+0000","last_change":"2024-09-05T02:00:12.783627+0000","last_active":"2024-09-05T02:01:48.307918+0000","last_peered":"2024-09-05T02:01:48.307918+0000","last_clean":"2024-09-05T02:01:48.307918+0000","last_became_active":"2024-09-05T02:00:12.783112+0000","last_became_peered":"2024-09-05T02:00:12.783112+0000","last_unstale":"2024-09-05T02:01:48.307918+0000","last_undegraded":"2024-09-05T02:01:48.307918+0000","last_fullsized":"2024-09-05T02:01:48.307918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:39:45.190311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508601+0000","last_change":"2024-09-05T02:00:21.413069+0000","last_active":"2024-09-05T02:01:26.508601+0000","last_peered":"2024-09-05T02:01:26.508601+0000","last_clean":"2024-09-05T02:01:26.508601+0000","last_became_active":"2024-09-05T02:00:21.412687+0000","last_became_peered":"2024-09-05T02:00:21.412687+0000","last_unstale":"2024-09-05T02:01:26.508601+0000","last_undegraded":"2024-09-05T02:01:26.508601+0000","last_fullsized":"2024-09-05T02:01:26.508601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:28.883071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268278+0000","last_change":"2024-09-05T02:00:14.363197+0000","last_active":"2024-09-05T02:01:26.268278+0000","last_peered":"2024-09-05T02:01:26.268278+0000","last_clean":"2024-09-05T02:01:26.268278+0000","last_became_active":"2024-09-05T02:00:14.362854+0000","last_became_peered":"2024-09-05T02:00:14.362854+0000","last_unstale":"2024-09-05T02:01:26.268278+0000","last_undegraded":"2024-09-05T02:01:26.268278+0000","last_fullsized":"2024-09-05T02:01:26.268278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:29:28.770604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269052+0000","last_change":"2024-09-05T02:00:12.348715+0000","last_active":"2024-09-05T02:01:26.269052+0000","last_peered":"2024-09-05T02:01:26.269052+0000","last_clean":"2024-09-05T02:01:26.269052+0000","last_became_active":"2024-09-05T02:00:12.348322+0000","last_became_peered":"2024-09-05T02:00:12.348322+0000","last_unstale":"2024-09-05T02:01:26.269052+0000","last_undegraded":"2024-09-05T02:01:26.269052+0000","last_fullsized":"2024-09-05T02:01:26.269052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:33:55.551080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.509548+0000","last_change":"2024-09-05T02:01:26.509548+0000","last_active":"2024-09-05T02:01:26.509548+0000","last_peered":"2024-09-05T02:01:26.509548+0000","last_clean":"2024-09-05T02:01:26.509548+0000","last_became_active":"2024-09-05T02:01:26.508993+0000","last_became_peered":"2024-09-05T02:01:26.508993+0000","last_unstale":"2024-09-05T02:01:26.509548+0000","last_undegraded":"2024-09-05T02:01:26.509548+0000","last_fullsized":"2024-09-05T02:01:26.509548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:46.109685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268767+0000","last_change":"2024-09-05T02:00:14.355969+0000","last_active":"2024-09-05T02:01:26.268767+0000","last_peered":"2024-09-05T02:01:26.268767+0000","last_clean":"2024-09-05T02:01:26.268767+0000","last_became_active":"2024-09-05T02:00:14.355581+0000","last_became_peered":"2024-09-05T02:00:14.355581+0000","last_unstale":"2024-09-05T02:01:26.268767+0000","last_undegraded":"2024-09-05T02:01:26.268767+0000","last_fullsized":"2024-09-05T02:01:26.268767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:55:22.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281299+0000","last_change":"2024-09-05T02:00:12.348148+0000","last_active":"2024-09-05T02:01:26.281299+0000","last_peered":"2024-09-05T02:01:26.281299+0000","last_clean":"2024-09-05T02:01:26.281299+0000","last_became_active":"2024-09-05T02:00:12.347857+0000","last_became_peered":"2024-09-05T02:00:12.347857+0000","last_unstale":"2024-09-05T02:01:26.281299+0000","last_undegraded":"2024-09-05T02:01:26.281299+0000","last_fullsized":"2024-09-05T02:01:26.281299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:22:21.368082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268247+0000","last_change":"2024-09-05T02:00:21.417476+0000","last_active":"2024-09-05T02:01:26.268247+0000","last_peered":"2024-09-05T02:01:26.268247+0000","last_clean":"2024-09-05T02:01:26.268247+0000","last_became_active":"2024-09-05T02:00:21.417036+0000","last_became_peered":"2024-09-05T02:00:21.417036+0000","last_unstale":"2024-09-05T02:01:26.268247+0000","last_undegraded":"2024-09-05T02:01:26.268247+0000","last_fullsized":"2024-09-05T02:01:26.268247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:15:22.510948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507726+0000","last_change":"2024-09-05T02:00:14.358542+0000","last_active":"2024-09-05T02:01:26.507726+0000","last_peered":"2024-09-05T02:01:26.507726+0000","last_clean":"2024-09-05T02:01:26.507726+0000","last_became_active":"2024-09-05T02:00:14.358228+0000","last_became_peered":"2024-09-05T02:00:14.358228+0000","last_unstale":"2024-09-05T02:01:26.507726+0000","last_undegraded":"2024-09-05T02:01:26.507726+0000","last_fullsized":"2024-09-05T02:01:26.507726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:01.627453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.270241+0000","last_change":"2024-09-05T02:00:12.345887+0000","last_active":"2024-09-05T02:01:26.270241+0000","last_peered":"2024-09-05T02:01:26.270241+0000","last_clean":"2024-09-05T02:01:26.270241+0000","last_became_active":"2024-09-05T02:00:12.345491+0000","last_became_peered":"2024-09-05T02:00:12.345491+0000","last_unstale":"2024-09-05T02:01:26.270241+0000","last_undegraded":"2024-09-05T02:01:26.270241+0000","last_fullsized":"2024-09-05T02:01:26.270241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:10:32.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268661+0000","last_change":"2024-09-05T02:00:21.418210+0000","last_active":"2024-09-05T02:01:26.268661+0000","last_peered":"2024-09-05T02:01:26.268661+0000","last_clean":"2024-09-05T02:01:26.268661+0000","last_became_active":"2024-09-05T02:00:21.417769+0000","last_became_peered":"2024-09-05T02:00:21.417769+0000","last_unstale":"2024-09-05T02:01:26.268661+0000","last_undegraded":"2024-09-05T02:01:26.268661+0000","last_fullsized":"2024-09-05T02:01:26.268661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:05.050499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269951+0000","last_change":"2024-09-05T02:00:14.367555+0000","last_active":"2024-09-05T02:01:26.269951+0000","last_peered":"2024-09-05T02:01:26.269951+0000","last_clean":"2024-09-05T02:01:26.269951+0000","last_became_active":"2024-09-05T02:00:14.367222+0000","last_became_peered":"2024-09-05T02:00:14.367222+0000","last_unstale":"2024-09-05T02:01:26.269951+0000","last_undegraded":"2024-09-05T02:01:26.269951+0000","last_fullsized":"2024-09-05T02:01:26.269951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:04:26.172141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.282875+0000","last_change":"2024-09-05T02:01:26.282875+0000","last_active":"2024-09-05T02:01:26.282875+0000","last_peered":"2024-09-05T02:01:26.282875+0000","last_clean":"2024-09-05T02:01:26.282875+0000","last_became_active":"2024-09-05T02:01:26.271901+0000","last_became_peered":"2024-09-05T02:01:26.271901+0000","last_unstale":"2024-09-05T02:01:26.282875+0000","last_undegraded":"2024-09-05T02:01:26.282875+0000","last_fullsized":"2024-09-05T02:01:26.282875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:29:45.232870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281297+0000","last_change":"2024-09-05T02:00:21.412374+0000","last_active":"2024-09-05T02:01:26.281297+0000","last_peered":"2024-09-05T02:01:26.281297+0000","last_clean":"2024-09-05T02:01:26.281297+0000","last_became_active":"2024-09-05T02:00:21.411926+0000","last_became_peered":"2024-09-05T02:00:21.411926+0000","last_unstale":"2024-09-05T02:01:26.281297+0000","last_undegraded":"2024-09-05T02:01:26.281297+0000","last_fullsized":"2024-09-05T02:01:26.281297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:38:29.657795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269814+0000","last_change":"2024-09-05T02:01:26.269814+0000","last_active":"2024-09-05T02:01:26.269814+0000","last_peered":"2024-09-05T02:01:26.269814+0000","last_clean":"2024-09-05T02:01:26.269814+0000","last_became_active":"2024-09-05T02:01:26.269320+0000","last_became_peered":"2024-09-05T02:01:26.269320+0000","last_unstale":"2024-09-05T02:01:26.269814+0000","last_undegraded":"2024-09-05T02:01:26.269814+0000","last_fullsized":"2024-09-05T02:01:26.269814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:04:21.161986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:47.305810+0000","last_change":"2024-09-05T02:00:12.349175+0000","last_active":"2024-09-05T02:01:47.305810+0000","last_peered":"2024-09-05T02:01:47.305810+0000","last_clean":"2024-09-05T02:01:47.305810+0000","last_became_active":"2024-09-05T02:00:12.348647+0000","last_became_peered":"2024-09-05T02:00:12.348647+0000","last_unstale":"2024-09-05T02:01:47.305810+0000","last_undegraded":"2024-09-05T02:01:47.305810+0000","last_fullsized":"2024-09-05T02:01:47.305810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:38:13.363462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265680+0000","last_change":"2024-09-05T02:00:21.414252+0000","last_active":"2024-09-05T02:01:26.265680+0000","last_peered":"2024-09-05T02:01:26.265680+0000","last_clean":"2024-09-05T02:01:26.265680+0000","last_became_active":"2024-09-05T02:00:21.412483+0000","last_became_peered":"2024-09-05T02:00:21.412483+0000","last_unstale":"2024-09-05T02:01:26.265680+0000","last_undegraded":"2024-09-05T02:01:26.265680+0000","last_fullsized":"2024-09-05T02:01:26.265680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:58.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268173+0000","last_change":"2024-09-05T02:00:14.361018+0000","last_active":"2024-09-05T02:01:26.268173+0000","last_peered":"2024-09-05T02:01:26.268173+0000","last_clean":"2024-09-05T02:01:26.268173+0000","last_became_active":"2024-09-05T02:00:14.360591+0000","last_became_peered":"2024-09-05T02:00:14.360591+0000","last_unstale":"2024-09-05T02:01:26.268173+0000","last_undegraded":"2024-09-05T02:01:26.268173+0000","last_fullsized":"2024-09-05T02:01:26.268173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:49.175681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268593+0000","last_change":"2024-09-05T02:00:12.353257+0000","last_active":"2024-09-05T02:01:26.268593+0000","last_peered":"2024-09-05T02:01:26.268593+0000","last_clean":"2024-09-05T02:01:26.268593+0000","last_became_active":"2024-09-05T02:00:12.352906+0000","last_became_peered":"2024-09-05T02:00:12.352906+0000","last_unstale":"2024-09-05T02:01:26.268593+0000","last_undegraded":"2024-09-05T02:01:26.268593+0000","last_fullsized":"2024-09-05T02:01:26.268593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:53:52.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508032+0000","last_change":"2024-09-05T02:00:21.410346+0000","last_active":"2024-09-05T02:01:26.508032+0000","last_peered":"2024-09-05T02:01:26.508032+0000","last_clean":"2024-09-05T02:01:26.508032+0000","last_became_active":"2024-09-05T02:00:21.409946+0000","last_became_peered":"2024-09-05T02:00:21.409946+0000","last_unstale":"2024-09-05T02:01:26.508032+0000","last_undegraded":"2024-09-05T02:01:26.508032+0000","last_fullsized":"2024-09-05T02:01:26.508032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:56.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.280907+0000","last_change":"2024-09-05T02:00:14.360265+0000","last_active":"2024-09-05T02:01:26.280907+0000","last_peered":"2024-09-05T02:01:26.280907+0000","last_clean":"2024-09-05T02:01:26.280907+0000","last_became_active":"2024-09-05T02:00:14.359958+0000","last_became_peered":"2024-09-05T02:00:14.359958+0000","last_unstale":"2024-09-05T02:01:26.280907+0000","last_undegraded":"2024-09-05T02:01:26.280907+0000","last_fullsized":"2024-09-05T02:01:26.280907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:15:45.211207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265890+0000","last_change":"2024-09-05T02:00:12.349443+0000","last_active":"2024-09-05T02:01:26.265890+0000","last_peered":"2024-09-05T02:01:26.265890+0000","last_clean":"2024-09-05T02:01:26.265890+0000","last_became_active":"2024-09-05T02:00:12.348988+0000","last_became_peered":"2024-09-05T02:00:12.348988+0000","last_unstale":"2024-09-05T02:01:26.265890+0000","last_undegraded":"2024-09-05T02:01:26.265890+0000","last_fullsized":"2024-09-05T02:01:26.265890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:14:21.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268391+0000","last_change":"2024-09-05T02:00:21.417662+0000","last_active":"2024-09-05T02:01:26.268391+0000","last_peered":"2024-09-05T02:01:26.268391+0000","last_clean":"2024-09-05T02:01:26.268391+0000","last_became_active":"2024-09-05T02:00:21.417238+0000","last_became_peered":"2024-09-05T02:00:21.417238+0000","last_unstale":"2024-09-05T02:01:26.268391+0000","last_undegraded":"2024-09-05T02:01:26.268391+0000","last_fullsized":"2024-09-05T02:01:26.268391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:05:59.304480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269560+0000","last_change":"2024-09-05T02:00:14.355928+0000","last_active":"2024-09-05T02:01:26.269560+0000","last_peered":"2024-09-05T02:01:26.269560+0000","last_clean":"2024-09-05T02:01:26.269560+0000","last_became_active":"2024-09-05T02:00:14.355576+0000","last_became_peered":"2024-09-05T02:00:14.355576+0000","last_unstale":"2024-09-05T02:01:26.269560+0000","last_undegraded":"2024-09-05T02:01:26.269560+0000","last_fullsized":"2024-09-05T02:01:26.269560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:22:50.095434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'34","reported_seq":177,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:48.307917+0000","last_change":"2024-09-05T02:00:12.352198+0000","last_active":"2024-09-05T02:01:48.307917+0000","last_peered":"2024-09-05T02:01:48.307917+0000","last_clean":"2024-09-05T02:01:48.307917+0000","last_became_active":"2024-09-05T02:00:12.351806+0000","last_became_peered":"2024-09-05T02:00:12.351806+0000","last_unstale":"2024-09-05T02:01:48.307917+0000","last_undegraded":"2024-09-05T02:01:48.307917+0000","last_fullsized":"2024-09-05T02:01:48.307917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.178160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":115,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.509575+0000","last_change":"2024-09-05T02:01:26.509575+0000","last_active":"2024-09-05T02:01:26.509575+0000","last_peered":"2024-09-05T02:01:26.509575+0000","last_clean":"2024-09-05T02:01:26.509575+0000","last_became_active":"2024-09-05T02:01:26.509108+0000","last_became_peered":"2024-09-05T02:01:26.509108+0000","last_unstale":"2024-09-05T02:01:26.509575+0000","last_undegraded":"2024-09-05T02:01:26.509575+0000","last_fullsized":"2024-09-05T02:01:26.509575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:55.503181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507570+0000","last_change":"2024-09-05T02:00:14.365372+0000","last_active":"2024-09-05T02:01:26.507570+0000","last_peered":"2024-09-05T02:01:26.507570+0000","last_clean":"2024-09-05T02:01:26.507570+0000","last_became_active":"2024-09-05T02:00:14.365064+0000","last_became_peered":"2024-09-05T02:00:14.365064+0000","last_unstale":"2024-09-05T02:01:26.507570+0000","last_undegraded":"2024-09-05T02:01:26.507570+0000","last_fullsized":"2024-09-05T02:01:26.507570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:02:29.378126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269261+0000","last_change":"2024-09-05T02:00:12.783487+0000","last_active":"2024-09-05T02:01:26.269261+0000","last_peered":"2024-09-05T02:01:26.269261+0000","last_clean":"2024-09-05T02:01:26.269261+0000","last_became_active":"2024-09-05T02:00:12.783087+0000","last_became_peered":"2024-09-05T02:00:12.783087+0000","last_unstale":"2024-09-05T02:01:26.269261+0000","last_undegraded":"2024-09-05T02:01:26.269261+0000","last_fullsized":"2024-09-05T02:01:26.269261+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:08.574845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.510122+0000","last_change":"2024-09-05T02:01:26.510122+0000","last_active":"2024-09-05T02:01:26.510122+0000","last_peered":"2024-09-05T02:01:26.510122+0000","last_clean":"2024-09-05T02:01:26.510122+0000","last_became_active":"2024-09-05T02:01:26.509427+0000","last_became_peered":"2024-09-05T02:01:26.509427+0000","last_unstale":"2024-09-05T02:01:26.510122+0000","last_undegraded":"2024-09-05T02:01:26.510122+0000","last_fullsized":"2024-09-05T02:01:26.510122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:50.139855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.272703+0000","last_change":"2024-09-05T02:01:26.272703+0000","last_active":"2024-09-05T02:01:26.272703+0000","last_peered":"2024-09-05T02:01:26.272703+0000","last_clean":"2024-09-05T02:01:26.272703+0000","last_became_active":"2024-09-05T02:01:26.272440+0000","last_became_peered":"2024-09-05T02:01:26.272440+0000","last_unstale":"2024-09-05T02:01:26.272703+0000","last_undegraded":"2024-09-05T02:01:26.272703+0000","last_fullsized":"2024-09-05T02:01:26.272703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:29.855027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268556+0000","last_change":"2024-09-05T02:00:12.349362+0000","last_active":"2024-09-05T02:01:26.268556+0000","last_peered":"2024-09-05T02:01:26.268556+0000","last_clean":"2024-09-05T02:01:26.268556+0000","last_became_active":"2024-09-05T02:00:12.349016+0000","last_became_peered":"2024-09-05T02:00:12.349016+0000","last_unstale":"2024-09-05T02:01:26.268556+0000","last_undegraded":"2024-09-05T02:01:26.268556+0000","last_fullsized":"2024-09-05T02:01:26.268556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:37.603717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265157+0000","last_change":"2024-09-05T02:00:21.412942+0000","last_active":"2024-09-05T02:01:26.265157+0000","last_peered":"2024-09-05T02:01:26.265157+0000","last_clean":"2024-09-05T02:01:26.265157+0000","last_became_active":"2024-09-05T02:00:21.412057+0000","last_became_peered":"2024-09-05T02:00:21.412057+0000","last_unstale":"2024-09-05T02:01:26.265157+0000","last_undegraded":"2024-09-05T02:01:26.265157+0000","last_fullsized":"2024-09-05T02:01:26.265157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:59.872818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507944+0000","last_change":"2024-09-05T02:00:14.356636+0000","last_active":"2024-09-05T02:01:26.507944+0000","last_peered":"2024-09-05T02:01:26.507944+0000","last_clean":"2024-09-05T02:01:26.507944+0000","last_became_active":"2024-09-05T02:00:14.356321+0000","last_became_peered":"2024-09-05T02:00:14.356321+0000","last_unstale":"2024-09-05T02:01:26.507944+0000","last_undegraded":"2024-09-05T02:01:26.507944+0000","last_fullsized":"2024-09-05T02:01:26.507944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:02.112646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":138,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:48.307387+0000","last_change":"2024-09-05T02:00:12.353078+0000","last_active":"2024-09-05T02:01:48.307387+0000","last_peered":"2024-09-05T02:01:48.307387+0000","last_clean":"2024-09-05T02:01:48.307387+0000","last_became_active":"2024-09-05T02:00:12.352707+0000","last_became_peered":"2024-09-05T02:00:12.352707+0000","last_unstale":"2024-09-05T02:01:48.307387+0000","last_undegraded":"2024-09-05T02:01:48.307387+0000","last_fullsized":"2024-09-05T02:01:48.307387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:28:29.934304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507633+0000","last_change":"2024-09-05T02:00:21.421362+0000","last_active":"2024-09-05T02:01:26.507633+0000","last_peered":"2024-09-05T02:01:26.507633+0000","last_clean":"2024-09-05T02:01:26.507633+0000","last_became_active":"2024-09-05T02:00:21.420998+0000","last_became_peered":"2024-09-05T02:00:21.420998+0000","last_unstale":"2024-09-05T02:01:26.507633+0000","last_undegraded":"2024-09-05T02:01:26.507633+0000","last_fullsized":"2024-09-05T02:01:26.507633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:26:14.105489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267672+0000","last_change":"2024-09-05T02:00:21.416703+0000","last_active":"2024-09-05T02:01:26.267672+0000","last_peered":"2024-09-05T02:01:26.267672+0000","last_clean":"2024-09-05T02:01:26.267672+0000","last_became_active":"2024-09-05T02:00:21.415913+0000","last_became_peered":"2024-09-05T02:00:21.415913+0000","last_unstale":"2024-09-05T02:01:26.267672+0000","last_undegraded":"2024-09-05T02:01:26.267672+0000","last_fullsized":"2024-09-05T02:01:26.267672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:56.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265232+0000","last_change":"2024-09-05T02:00:14.355658+0000","last_active":"2024-09-05T02:01:26.265232+0000","last_peered":"2024-09-05T02:01:26.265232+0000","last_clean":"2024-09-05T02:01:26.265232+0000","last_became_active":"2024-09-05T02:00:14.355087+0000","last_became_peered":"2024-09-05T02:00:14.355087+0000","last_unstale":"2024-09-05T02:01:26.265232+0000","last_undegraded":"2024-09-05T02:01:26.265232+0000","last_fullsized":"2024-09-05T02:01:26.265232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:58:08.369027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269514+0000","last_change":"2024-09-05T02:00:12.348171+0000","last_active":"2024-09-05T02:01:26.269514+0000","last_peered":"2024-09-05T02:01:26.269514+0000","last_clean":"2024-09-05T02:01:26.269514+0000","last_became_active":"2024-09-05T02:00:12.347760+0000","last_became_peered":"2024-09-05T02:00:12.347760+0000","last_unstale":"2024-09-05T02:01:26.269514+0000","last_undegraded":"2024-09-05T02:01:26.269514+0000","last_fullsized":"2024-09-05T02:01:26.269514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:03:43.748458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508807+0000","last_change":"2024-09-05T02:00:21.413015+0000","last_active":"2024-09-05T02:01:26.508807+0000","last_peered":"2024-09-05T02:01:26.508807+0000","last_clean":"2024-09-05T02:01:26.508807+0000","last_became_active":"2024-09-05T02:00:21.412631+0000","last_became_peered":"2024-09-05T02:00:21.412631+0000","last_unstale":"2024-09-05T02:01:26.508807+0000","last_undegraded":"2024-09-05T02:01:26.508807+0000","last_fullsized":"2024-09-05T02:01:26.508807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:35.094273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.506938+0000","last_change":"2024-09-05T02:00:14.354552+0000","last_active":"2024-09-05T02:01:26.506938+0000","last_peered":"2024-09-05T02:01:26.506938+0000","last_clean":"2024-09-05T02:01:26.506938+0000","last_became_active":"2024-09-05T02:00:14.354150+0000","last_became_peered":"2024-09-05T02:00:14.354150+0000","last_unstale":"2024-09-05T02:01:26.506938+0000","last_undegraded":"2024-09-05T02:01:26.506938+0000","last_fullsized":"2024-09-05T02:01:26.506938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:16:19.836148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.280777+0000","last_change":"2024-09-05T02:00:12.342608+0000","last_active":"2024-09-05T02:01:26.280777+0000","last_peered":"2024-09-05T02:01:26.280777+0000","last_clean":"2024-09-05T02:01:26.280777+0000","last_became_active":"2024-09-05T02:00:12.342235+0000","last_became_peered":"2024-09-05T02:00:12.342235+0000","last_unstale":"2024-09-05T02:01:26.280777+0000","last_undegraded":"2024-09-05T02:01:26.280777+0000","last_fullsized":"2024-09-05T02:01:26.280777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:54.568963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265875+0000","last_change":"2024-09-05T02:00:12.352247+0000","last_active":"2024-09-05T02:01:26.265875+0000","last_peered":"2024-09-05T02:01:26.265875+0000","last_clean":"2024-09-05T02:01:26.265875+0000","last_became_active":"2024-09-05T02:00:12.351897+0000","last_became_peered":"2024-09-05T02:00:12.351897+0000","last_unstale":"2024-09-05T02:01:26.265875+0000","last_undegraded":"2024-09-05T02:01:26.265875+0000","last_fullsized":"2024-09-05T02:01:26.265875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:48.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268681+0000","last_change":"2024-09-05T02:00:14.366415+0000","last_active":"2024-09-05T02:01:26.268681+0000","last_peered":"2024-09-05T02:01:26.268681+0000","last_clean":"2024-09-05T02:01:26.268681+0000","last_became_active":"2024-09-05T02:00:14.366103+0000","last_became_peered":"2024-09-05T02:00:14.366103+0000","last_unstale":"2024-09-05T02:01:26.268681+0000","last_undegraded":"2024-09-05T02:01:26.268681+0000","last_fullsized":"2024-09-05T02:01:26.268681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:22:21.585867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265920+0000","last_change":"2024-09-05T02:00:21.416879+0000","last_active":"2024-09-05T02:01:26.265920+0000","last_peered":"2024-09-05T02:01:26.265920+0000","last_clean":"2024-09-05T02:01:26.265920+0000","last_became_active":"2024-09-05T02:00:21.415887+0000","last_became_peered":"2024-09-05T02:00:21.415887+0000","last_unstale":"2024-09-05T02:01:26.265920+0000","last_undegraded":"2024-09-05T02:01:26.265920+0000","last_fullsized":"2024-09-05T02:01:26.265920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:58.106529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:48.308280+0000","last_change":"2024-09-05T02:00:12.354055+0000","last_active":"2024-09-05T02:01:48.308280+0000","last_peered":"2024-09-05T02:01:48.308280+0000","last_clean":"2024-09-05T02:01:48.308280+0000","last_became_active":"2024-09-05T02:00:12.353754+0000","last_became_peered":"2024-09-05T02:00:12.353754+0000","last_unstale":"2024-09-05T02:01:48.308280+0000","last_undegraded":"2024-09-05T02:01:48.308280+0000","last_fullsized":"2024-09-05T02:01:48.308280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:17.671079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269072+0000","last_change":"2024-09-05T02:00:14.358646+0000","last_active":"2024-09-05T02:01:26.269072+0000","last_peered":"2024-09-05T02:01:26.269072+0000","last_clean":"2024-09-05T02:01:26.269072+0000","last_became_active":"2024-09-05T02:00:14.358337+0000","last_became_peered":"2024-09-05T02:00:14.358337+0000","last_unstale":"2024-09-05T02:01:26.269072+0000","last_undegraded":"2024-09-05T02:01:26.269072+0000","last_fullsized":"2024-09-05T02:01:26.269072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:20:39.976731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.266492+0000","last_change":"2024-09-05T02:00:21.416611+0000","last_active":"2024-09-05T02:01:26.266492+0000","last_peered":"2024-09-05T02:01:26.266492+0000","last_clean":"2024-09-05T02:01:26.266492+0000","last_became_active":"2024-09-05T02:00:21.416152+0000","last_became_peered":"2024-09-05T02:00:21.416152+0000","last_unstale":"2024-09-05T02:01:26.266492+0000","last_undegraded":"2024-09-05T02:01:26.266492+0000","last_fullsized":"2024-09-05T02:01:26.266492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:20.670383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507763+0000","last_change":"2024-09-05T02:00:14.359015+0000","last_active":"2024-09-05T02:01:26.507763+0000","last_peered":"2024-09-05T02:01:26.507763+0000","last_clean":"2024-09-05T02:01:26.507763+0000","last_became_active":"2024-09-05T02:00:14.358642+0000","last_became_peered":"2024-09-05T02:00:14.358642+0000","last_unstale":"2024-09-05T02:01:26.507763+0000","last_undegraded":"2024-09-05T02:01:26.507763+0000","last_fullsized":"2024-09-05T02:01:26.507763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:18:03.723033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265712+0000","last_change":"2024-09-05T02:00:14.362391+0000","last_active":"2024-09-05T02:01:26.265712+0000","last_peered":"2024-09-05T02:01:26.265712+0000","last_clean":"2024-09-05T02:01:26.265712+0000","last_became_active":"2024-09-05T02:00:14.362007+0000","last_became_peered":"2024-09-05T02:00:14.362007+0000","last_unstale":"2024-09-05T02:01:26.265712+0000","last_undegraded":"2024-09-05T02:01:26.265712+0000","last_fullsized":"2024-09-05T02:01:26.265712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:39:02.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269863+0000","last_change":"2024-09-05T02:01:26.269863+0000","last_active":"2024-09-05T02:01:26.269863+0000","last_peered":"2024-09-05T02:01:26.269863+0000","last_clean":"2024-09-05T02:01:26.269863+0000","last_became_active":"2024-09-05T02:01:26.269451+0000","last_became_peered":"2024-09-05T02:01:26.269451+0000","last_unstale":"2024-09-05T02:01:26.269863+0000","last_undegraded":"2024-09-05T02:01:26.269863+0000","last_fullsized":"2024-09-05T02:01:26.269863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:11.900726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267767+0000","last_change":"2024-09-05T02:00:14.357820+0000","last_active":"2024-09-05T02:01:26.267767+0000","last_peered":"2024-09-05T02:01:26.267767+0000","last_clean":"2024-09-05T02:01:26.267767+0000","last_became_active":"2024-09-05T02:00:14.357320+0000","last_became_peered":"2024-09-05T02:00:14.357320+0000","last_unstale":"2024-09-05T02:01:26.267767+0000","last_undegraded":"2024-09-05T02:01:26.267767+0000","last_fullsized":"2024-09-05T02:01:26.267767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:39.105192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281273+0000","last_change":"2024-09-05T02:00:14.356726+0000","last_active":"2024-09-05T02:01:26.281273+0000","last_peered":"2024-09-05T02:01:26.281273+0000","last_clean":"2024-09-05T02:01:26.281273+0000","last_became_active":"2024-09-05T02:00:14.356452+0000","last_became_peered":"2024-09-05T02:00:14.356452+0000","last_unstale":"2024-09-05T02:01:26.281273+0000","last_undegraded":"2024-09-05T02:01:26.281273+0000","last_fullsized":"2024-09-05T02:01:26.281273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:34:36.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507691+0000","last_change":"2024-09-05T02:00:14.357645+0000","last_active":"2024-09-05T02:01:26.507691+0000","last_peered":"2024-09-05T02:01:26.507691+0000","last_clean":"2024-09-05T02:01:26.507691+0000","last_became_active":"2024-09-05T02:00:14.357330+0000","last_became_peered":"2024-09-05T02:00:14.357330+0000","last_unstale":"2024-09-05T02:01:26.507691+0000","last_undegraded":"2024-09-05T02:01:26.507691+0000","last_fullsized":"2024-09-05T02:01:26.507691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:42.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267354+0000","last_change":"2024-09-05T02:00:14.363413+0000","last_active":"2024-09-05T02:01:26.267354+0000","last_peered":"2024-09-05T02:01:26.267354+0000","last_clean":"2024-09-05T02:01:26.267354+0000","last_became_active":"2024-09-05T02:00:14.362821+0000","last_became_peered":"2024-09-05T02:00:14.362821+0000","last_unstale":"2024-09-05T02:01:26.267354+0000","last_undegraded":"2024-09-05T02:01:26.267354+0000","last_fullsized":"2024-09-05T02:01:26.267354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:49:39.084171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507154+0000","last_change":"2024-09-05T02:00:14.366581+0000","last_active":"2024-09-05T02:01:26.507154+0000","last_peered":"2024-09-05T02:01:26.507154+0000","last_clean":"2024-09-05T02:01:26.507154+0000","last_became_active":"2024-09-05T02:00:14.366287+0000","last_became_peered":"2024-09-05T02:00:14.366287+0000","last_unstale":"2024-09-05T02:01:26.507154+0000","last_undegraded":"2024-09-05T02:01:26.507154+0000","last_fullsized":"2024-09-05T02:01:26.507154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:36:57.143489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268446+0000","last_change":"2024-09-05T02:00:14.360311+0000","last_active":"2024-09-05T02:01:26.268446+0000","last_peered":"2024-09-05T02:01:26.268446+0000","last_clean":"2024-09-05T02:01:26.268446+0000","last_became_active":"2024-09-05T02:00:14.360028+0000","last_became_peered":"2024-09-05T02:00:14.360028+0000","last_unstale":"2024-09-05T02:01:26.268446+0000","last_undegraded":"2024-09-05T02:01:26.268446+0000","last_fullsized":"2024-09-05T02:01:26.268446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:35:38.804555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.506952+0000","last_change":"2024-09-05T02:00:14.365853+0000","last_active":"2024-09-05T02:01:26.506952+0000","last_peered":"2024-09-05T02:01:26.506952+0000","last_clean":"2024-09-05T02:01:26.506952+0000","last_became_active":"2024-09-05T02:00:14.365549+0000","last_became_peered":"2024-09-05T02:00:14.365549+0000","last_unstale":"2024-09-05T02:01:26.506952+0000","last_undegraded":"2024-09-05T02:01:26.506952+0000","last_fullsized":"2024-09-05T02:01:26.506952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:58:45.897592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507532+0000","last_change":"2024-09-05T02:00:14.367357+0000","last_active":"2024-09-05T02:01:26.507532+0000","last_peered":"2024-09-05T02:01:26.507532+0000","last_clean":"2024-09-05T02:01:26.507532+0000","last_became_active":"2024-09-05T02:00:14.366794+0000","last_became_peered":"2024-09-05T02:00:14.366794+0000","last_unstale":"2024-09-05T02:01:26.507532+0000","last_undegraded":"2024-09-05T02:01:26.507532+0000","last_fullsized":"2024-09-05T02:01:26.507532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:40:51.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281146+0000","last_change":"2024-09-05T02:00:14.356991+0000","last_active":"2024-09-05T02:01:26.281146+0000","last_peered":"2024-09-05T02:01:26.281146+0000","last_clean":"2024-09-05T02:01:26.281146+0000","last_became_active":"2024-09-05T02:00:14.356662+0000","last_became_peered":"2024-09-05T02:00:14.356662+0000","last_unstale":"2024-09-05T02:01:26.281146+0000","last_undegraded":"2024-09-05T02:01:26.281146+0000","last_fullsized":"2024-09-05T02:01:26.281146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:39:53.087448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507629+0000","last_change":"2024-09-05T02:00:14.357750+0000","last_active":"2024-09-05T02:01:26.507629+0000","last_peered":"2024-09-05T02:01:26.507629+0000","last_clean":"2024-09-05T02:01:26.507629+0000","last_became_active":"2024-09-05T02:00:14.357413+0000","last_became_peered":"2024-09-05T02:00:14.357413+0000","last_unstale":"2024-09-05T02:01:26.507629+0000","last_undegraded":"2024-09-05T02:01:26.507629+0000","last_fullsized":"2024-09-05T02:01:26.507629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:29:47.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.266873+0000","last_change":"2024-09-05T02:00:14.364068+0000","last_active":"2024-09-05T02:01:26.266873+0000","last_peered":"2024-09-05T02:01:26.266873+0000","last_clean":"2024-09-05T02:01:26.266873+0000","last_became_active":"2024-09-05T02:00:14.363674+0000","last_became_peered":"2024-09-05T02:00:14.363674+0000","last_unstale":"2024-09-05T02:01:26.266873+0000","last_undegraded":"2024-09-05T02:01:26.266873+0000","last_fullsized":"2024-09-05T02:01:26.266873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:28:00.788668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265726+0000","last_change":"2024-09-05T02:00:14.355152+0000","last_active":"2024-09-05T02:01:26.265726+0000","last_peered":"2024-09-05T02:01:26.265726+0000","last_clean":"2024-09-05T02:01:26.265726+0000","last_became_active":"2024-09-05T02:00:14.354874+0000","last_became_peered":"2024-09-05T02:00:14.354874+0000","last_unstale":"2024-09-05T02:01:26.265726+0000","last_undegraded":"2024-09-05T02:01:26.265726+0000","last_fullsized":"2024-09-05T02:01:26.265726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:14:17.069049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508040+0000","last_change":"2024-09-05T02:00:14.360241+0000","last_active":"2024-09-05T02:01:26.508040+0000","last_peered":"2024-09-05T02:01:26.508040+0000","last_clean":"2024-09-05T02:01:26.508040+0000","last_became_active":"2024-09-05T02:00:14.359648+0000","last_became_peered":"2024-09-05T02:00:14.359648+0000","last_unstale":"2024-09-05T02:01:26.508040+0000","last_undegraded":"2024-09-05T02:01:26.508040+0000","last_fullsized":"2024-09-05T02:01:26.508040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:15.060575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507514+0000","last_change":"2024-09-05T02:00:14.366094+0000","last_active":"2024-09-05T02:01:26.507514+0000","last_peered":"2024-09-05T02:01:26.507514+0000","last_clean":"2024-09-05T02:01:26.507514+0000","last_became_active":"2024-09-05T02:00:14.365788+0000","last_became_peered":"2024-09-05T02:00:14.365788+0000","last_unstale":"2024-09-05T02:01:26.507514+0000","last_undegraded":"2024-09-05T02:01:26.507514+0000","last_fullsized":"2024-09-05T02:01:26.507514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:23:00.909487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267635+0000","last_change":"2024-09-05T02:00:14.364651+0000","last_active":"2024-09-05T02:01:26.267635+0000","last_peered":"2024-09-05T02:01:26.267635+0000","last_clean":"2024-09-05T02:01:26.267635+0000","last_became_active":"2024-09-05T02:00:14.364365+0000","last_became_peered":"2024-09-05T02:00:14.364365+0000","last_unstale":"2024-09-05T02:01:26.267635+0000","last_undegraded":"2024-09-05T02:01:26.267635+0000","last_fullsized":"2024-09-05T02:01:26.267635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:14.978382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267589+0000","last_change":"2024-09-05T02:00:14.364761+0000","last_active":"2024-09-05T02:01:26.267589+0000","last_peered":"2024-09-05T02:01:26.267589+0000","last_clean":"2024-09-05T02:01:26.267589+0000","last_became_active":"2024-09-05T02:00:14.364204+0000","last_became_peered":"2024-09-05T02:00:14.364204+0000","last_unstale":"2024-09-05T02:01:26.267589+0000","last_undegraded":"2024-09-05T02:01:26.267589+0000","last_fullsized":"2024-09-05T02:01:26.267589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:16:51.076304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281131+0000","last_change":"2024-09-05T02:00:14.357632+0000","last_active":"2024-09-05T02:01:26.281131+0000","last_peered":"2024-09-05T02:01:26.281131+0000","last_clean":"2024-09-05T02:01:26.281131+0000","last_became_active":"2024-09-05T02:00:14.357348+0000","last_became_peered":"2024-09-05T02:00:14.357348+0000","last_unstale":"2024-09-05T02:01:26.281131+0000","last_undegraded":"2024-09-05T02:01:26.281131+0000","last_fullsized":"2024-09-05T02:01:26.281131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:11.163292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268194+0000","last_change":"2024-09-05T02:00:14.361484+0000","last_active":"2024-09-05T02:01:26.268194+0000","last_peered":"2024-09-05T02:01:26.268194+0000","last_clean":"2024-09-05T02:01:26.268194+0000","last_became_active":"2024-09-05T02:00:14.361156+0000","last_became_peered":"2024-09-05T02:00:14.361156+0000","last_unstale":"2024-09-05T02:01:26.268194+0000","last_undegraded":"2024-09-05T02:01:26.268194+0000","last_fullsized":"2024-09-05T02:01:26.268194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:40.124733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268635+0000","last_change":"2024-09-05T02:00:14.358325+0000","last_active":"2024-09-05T02:01:26.268635+0000","last_peered":"2024-09-05T02:01:26.268635+0000","last_clean":"2024-09-05T02:01:26.268635+0000","last_became_active":"2024-09-05T02:00:14.357993+0000","last_became_peered":"2024-09-05T02:00:14.357993+0000","last_unstale":"2024-09-05T02:01:26.268635+0000","last_undegraded":"2024-09-05T02:01:26.268635+0000","last_fullsized":"2024-09-05T02:01:26.268635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:18.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.266781+0000","last_change":"2024-09-05T02:00:14.365291+0000","last_active":"2024-09-05T02:01:26.266781+0000","last_peered":"2024-09-05T02:01:26.266781+0000","last_clean":"2024-09-05T02:01:26.266781+0000","last_became_active":"2024-09-05T02:00:14.364937+0000","last_became_peered":"2024-09-05T02:00:14.364937+0000","last_unstale":"2024-09-05T02:01:26.266781+0000","last_undegraded":"2024-09-05T02:01:26.266781+0000","last_fullsized":"2024-09-05T02:01:26.266781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:42:57.712651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508067+0000","last_change":"2024-09-05T02:00:14.367603+0000","last_active":"2024-09-05T02:01:26.508067+0000","last_peered":"2024-09-05T02:01:26.508067+0000","last_clean":"2024-09-05T02:01:26.508067+0000","last_became_active":"2024-09-05T02:00:14.367267+0000","last_became_peered":"2024-09-05T02:00:14.367267+0000","last_unstale":"2024-09-05T02:01:26.508067+0000","last_undegraded":"2024-09-05T02:01:26.508067+0000","last_fullsized":"2024-09-05T02:01:26.508067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:51:34.595898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.506938+0000","last_change":"2024-09-05T02:00:14.369499+0000","last_active":"2024-09-05T02:01:26.506938+0000","last_peered":"2024-09-05T02:01:26.506938+0000","last_clean":"2024-09-05T02:01:26.506938+0000","last_became_active":"2024-09-05T02:00:14.369169+0000","last_became_peered":"2024-09-05T02:00:14.369169+0000","last_unstale":"2024-09-05T02:01:26.506938+0000","last_undegraded":"2024-09-05T02:01:26.506938+0000","last_fullsized":"2024-09-05T02:01:26.506938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:17.161204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267502+0000","last_change":"2024-09-05T02:00:14.360320+0000","last_active":"2024-09-05T02:01:26.267502+0000","last_peered":"2024-09-05T02:01:26.267502+0000","last_clean":"2024-09-05T02:01:26.267502+0000","last_became_active":"2024-09-05T02:00:14.359991+0000","last_became_peered":"2024-09-05T02:00:14.359991+0000","last_unstale":"2024-09-05T02:01:26.267502+0000","last_undegraded":"2024-09-05T02:01:26.267502+0000","last_fullsized":"2024-09-05T02:01:26.267502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:03.721863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265823+0000","last_change":"2024-09-05T02:00:14.364660+0000","last_active":"2024-09-05T02:01:26.265823+0000","last_peered":"2024-09-05T02:01:26.265823+0000","last_clean":"2024-09-05T02:01:26.265823+0000","last_became_active":"2024-09-05T02:00:14.364313+0000","last_became_peered":"2024-09-05T02:00:14.364313+0000","last_unstale":"2024-09-05T02:01:26.265823+0000","last_undegraded":"2024-09-05T02:01:26.265823+0000","last_fullsized":"2024-09-05T02:01:26.265823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:47.496063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268892+0000","last_change":"2024-09-05T02:00:14.359853+0000","last_active":"2024-09-05T02:01:26.268892+0000","last_peered":"2024-09-05T02:01:26.268892+0000","last_clean":"2024-09-05T02:01:26.268892+0000","last_became_active":"2024-09-05T02:00:14.359543+0000","last_became_peered":"2024-09-05T02:00:14.359543+0000","last_unstale":"2024-09-05T02:01:26.268892+0000","last_undegraded":"2024-09-05T02:01:26.268892+0000","last_fullsized":"2024-09-05T02:01:26.268892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:30:29.415492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.270888+0000","last_change":"2024-09-05T02:01:26.270888+0000","last_active":"2024-09-05T02:01:26.270888+0000","last_peered":"2024-09-05T02:01:26.270888+0000","last_clean":"2024-09-05T02:01:26.270888+0000","last_became_active":"2024-09-05T02:01:26.270607+0000","last_became_peered":"2024-09-05T02:01:26.270607+0000","last_unstale":"2024-09-05T02:01:26.270888+0000","last_undegraded":"2024-09-05T02:01:26.270888+0000","last_fullsized":"2024-09-05T02:01:26.270888+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:16.563521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267574+0000","last_change":"2024-09-05T02:00:14.367191+0000","last_active":"2024-09-05T02:01:26.267574+0000","last_peered":"2024-09-05T02:01:26.267574+0000","last_clean":"2024-09-05T02:01:26.267574+0000","last_became_active":"2024-09-05T02:00:14.366831+0000","last_became_peered":"2024-09-05T02:00:14.366831+0000","last_unstale":"2024-09-05T02:01:26.267574+0000","last_undegraded":"2024-09-05T02:01:26.267574+0000","last_fullsized":"2024-09-05T02:01:26.267574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:55.690642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267770+0000","last_change":"2024-09-05T02:00:14.355774+0000","last_active":"2024-09-05T02:01:26.267770+0000","last_peered":"2024-09-05T02:01:26.267770+0000","last_clean":"2024-09-05T02:01:26.267770+0000","last_became_active":"2024-09-05T02:00:14.355293+0000","last_became_peered":"2024-09-05T02:00:14.355293+0000","last_unstale":"2024-09-05T02:01:26.267770+0000","last_undegraded":"2024-09-05T02:01:26.267770+0000","last_fullsized":"2024-09-05T02:01:26.267770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:07:00.547820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508355+0000","last_change":"2024-09-05T02:00:14.359978+0000","last_active":"2024-09-05T02:01:26.508355+0000","last_peered":"2024-09-05T02:01:26.508355+0000","last_clean":"2024-09-05T02:01:26.508355+0000","last_became_active":"2024-09-05T02:00:14.359209+0000","last_became_peered":"2024-09-05T02:00:14.359209+0000","last_unstale":"2024-09-05T02:01:26.508355+0000","last_undegraded":"2024-09-05T02:01:26.508355+0000","last_fullsized":"2024-09-05T02:01:26.508355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:27.848906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69284,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":540,"num_read_kb":346,"num_write":249,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579937,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1007616,"data_stored":497661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808870,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1699840,"data_stored":1146160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":501294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201329,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":497576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430260,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1626112,"data_stored":1094075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":492702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36948,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708204,"statfs":{"total":95995035648,"available":95957200896,"internally_reserved":0,"allocated":1634304,"data_stored":1089657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1024000,"data_stored":496888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1019904,"data_stored":492410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1085440,"data_stored":556702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1073152,"data_stored":555808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":494551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T02:01:52.913 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T02:01:53.178 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:53.509 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:53 smithi103 ceph-mon[30902]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:53.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:53 smithi167 ceph-mon[35122]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:53.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:53 smithi139 ceph-mon[34262]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:53.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.709+0000 7f9c8cb12640 1 -- 172.21.15.103:0/95578640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 msgr2=0x7f9c8810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:53.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.709+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/95578640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 0x7f9c8810d940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009cd0 tx=0x7f9c8002a8e0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.709+0000 7f9c8cb12640 1 -- 172.21.15.103:0/95578640 shutdown_connections 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.709+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/95578640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 0x7f9c8810d940 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.709+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/95578640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c88100a70 0x7f9c88100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.709+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/95578640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c88106a70 0x7f9c88106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.709+0000 7f9c8cb12640 1 -- 172.21.15.103:0/95578640 >> 172.21.15.103:0/95578640 conn(0x7f9c880fc790 msgr2=0x7f9c880febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.709+0000 7f9c8cb12640 1 -- 172.21.15.103:0/95578640 shutdown_connections 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.709+0000 7f9c8cb12640 1 -- 172.21.15.103:0/95578640 wait complete. 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c8cb12640 1 Processor -- start 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c8cb12640 1 -- start start 2024-09-05T02:01:53.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c8cb12640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c88100a70 0x7f9c881961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c8cb12640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 0x7f9c881966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c8cb12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c88106a70 0x7f9c8819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c8cb12640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8810fc30 con 0x7f9c88101410 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c8cb12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8810fab0 con 0x7f9c88106a70 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c8cb12640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8810fdb0 con 0x7f9c88100a70 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c85d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 0x7f9c881966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c85d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 0x7f9c881966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60740/0 (socket says 172.21.15.103:60740) 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c85d9b640 1 -- 172.21.15.103:0/1204636207 learned_addr learned my addr 172.21.15.103:0/1204636207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c8659c640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c88100a70 0x7f9c881961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c86d9d640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c88106a70 0x7f9c8819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c85d9b640 1 -- 172.21.15.103:0/1204636207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c88100a70 msgr2=0x7f9c881961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c85d9b640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c88100a70 0x7f9c881961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c85d9b640 1 -- 172.21.15.103:0/1204636207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c88106a70 msgr2=0x7f9c8819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c85d9b640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c88106a70 0x7f9c8819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.710+0000 7f9c85d9b640 1 -- 172.21.15.103:0/1204636207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8819b160 con 0x7f9c88101410 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.711+0000 7f9c85d9b640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 0x7f9c881966f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800b4a0 tx=0x7f9c7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.711+0000 7f9c7f7fe640 1 -- 172.21.15.103:0/1204636207 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78014070 con 0x7f9c88101410 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.711+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8819b450 con 0x7f9c88101410 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.711+0000 7f9c7f7fe640 1 -- 172.21.15.103:0/1204636207 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c78010ac0 con 0x7f9c88101410 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.711+0000 7f9c7f7fe640 1 -- 172.21.15.103:0/1204636207 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78017c30 con 0x7f9c88101410 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.711+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c881a2d30 con 0x7f9c88101410 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.712+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c88100ed0 con 0x7f9c88101410 2024-09-05T02:01:53.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.713+0000 7f9c7f7fe640 1 -- 172.21.15.103:0/1204636207 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9c78017070 con 0x7f9c88101410 2024-09-05T02:01:53.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.714+0000 7f9c7f7fe640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9c740a4e20 0x7f9c740a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:53.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.714+0000 7f9c8659c640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9c740a4e20 0x7f9c740a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:53.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.714+0000 7f9c7f7fe640 1 -- 172.21.15.103:0/1204636207 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f9c780ce8c0 con 0x7f9c88101410 2024-09-05T02:01:53.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.714+0000 7f9c8659c640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9c740a4e20 0x7f9c740a72e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c700058c0 tx=0x7f9c7000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:53.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.718+0000 7f9c7f7fe640 1 -- 172.21.15.103:0/1204636207 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c7809b470 con 0x7f9c88101410 2024-09-05T02:01:53.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.881+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c88062630 con 0x7f9c740a4e20 2024-09-05T02:01:53.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.889+0000 7f9c7f7fe640 1 -- 172.21.15.103:0/1204636207 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340395 (secure 0 0 0) 0x7f9c88062630 con 0x7f9c740a4e20 2024-09-05T02:01:53.894 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:53.895 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-05T02:01:53.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.892+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9c740a4e20 msgr2=0x7f9c740a72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:53.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.892+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9c740a4e20 0x7f9c740a72e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c700058c0 tx=0x7f9c7000a4c0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.892+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 msgr2=0x7f9c881966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:53.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.892+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 0x7f9c881966f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800b4a0 tx=0x7f9c7800b960 comp rx=0 tx=0).stop 2024-09-05T02:01:53.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.893+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 shutdown_connections 2024-09-05T02:01:53.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.893+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c88106a70 0x7f9c8819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.893+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c88101410 0x7f9c881966f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.893+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9c740a4e20 0x7f9c740a72e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.893+0000 7f9c8cb12640 1 --2- 172.21.15.103:0/1204636207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c88100a70 0x7f9c881961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:53.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.893+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 >> 172.21.15.103:0/1204636207 conn(0x7f9c880fc790 msgr2=0x7f9c880fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:53.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.893+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 shutdown_connections 2024-09-05T02:01:53.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:53.893+0000 7f9c8cb12640 1 -- 172.21.15.103:0/1204636207 wait complete. 2024-09-05T02:01:54.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:54 smithi103 ceph-mon[30902]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:01:54.609 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-05T02:01:53.106252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659886,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":671,"num_read_kb":455,"num_write":432,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392384,"kb_used_data":13940,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549440,"statfs":{"total":1151940427776,"available":1151538626560,"internally_reserved":0,"allocated":14274560,"data_stored":7915484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003075"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:47.305002+0000","last_change":"2024-09-05T02:00:12.347638+0000","last_active":"2024-09-05T02:01:47.305002+0000","last_peered":"2024-09-05T02:01:47.305002+0000","last_clean":"2024-09-05T02:01:47.305002+0000","last_became_active":"2024-09-05T02:00:12.347314+0000","last_became_peered":"2024-09-05T02:00:12.347314+0000","last_unstale":"2024-09-05T02:01:47.305002+0000","last_undegraded":"2024-09-05T02:01:47.305002+0000","last_fullsized":"2024-09-05T02:01:47.305002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:08:39.084117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507253+0000","last_change":"2024-09-05T02:00:14.360262+0000","last_active":"2024-09-05T02:01:26.507253+0000","last_peered":"2024-09-05T02:01:26.507253+0000","last_clean":"2024-09-05T02:01:26.507253+0000","last_became_active":"2024-09-05T02:00:14.359965+0000","last_became_peered":"2024-09-05T02:00:14.359965+0000","last_unstale":"2024-09-05T02:01:26.507253+0000","last_undegraded":"2024-09-05T02:01:26.507253+0000","last_fullsized":"2024-09-05T02:01:26.507253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:01:56.109169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265425+0000","last_change":"2024-09-05T02:00:21.410692+0000","last_active":"2024-09-05T02:01:26.265425+0000","last_peered":"2024-09-05T02:01:26.265425+0000","last_clean":"2024-09-05T02:01:26.265425+0000","last_became_active":"2024-09-05T02:00:21.410298+0000","last_became_peered":"2024-09-05T02:00:21.410298+0000","last_unstale":"2024-09-05T02:01:26.265425+0000","last_undegraded":"2024-09-05T02:01:26.265425+0000","last_fullsized":"2024-09-05T02:01:26.265425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:25:31.241700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507935+0000","last_change":"2024-09-05T02:00:12.783109+0000","last_active":"2024-09-05T02:01:26.507935+0000","last_peered":"2024-09-05T02:01:26.507935+0000","last_clean":"2024-09-05T02:01:26.507935+0000","last_became_active":"2024-09-05T02:00:12.782773+0000","last_became_peered":"2024-09-05T02:00:12.782773+0000","last_unstale":"2024-09-05T02:01:26.507935+0000","last_undegraded":"2024-09-05T02:01:26.507935+0000","last_fullsized":"2024-09-05T02:01:26.507935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:29:12.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267769+0000","last_change":"2024-09-05T02:00:14.370425+0000","last_active":"2024-09-05T02:01:26.267769+0000","last_peered":"2024-09-05T02:01:26.267769+0000","last_clean":"2024-09-05T02:01:26.267769+0000","last_became_active":"2024-09-05T02:00:14.370102+0000","last_became_peered":"2024-09-05T02:00:14.370102+0000","last_unstale":"2024-09-05T02:01:26.267769+0000","last_undegraded":"2024-09-05T02:01:26.267769+0000","last_fullsized":"2024-09-05T02:01:26.267769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:22:43.837418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268987+0000","last_change":"2024-09-05T02:00:21.413082+0000","last_active":"2024-09-05T02:01:26.268987+0000","last_peered":"2024-09-05T02:01:26.268987+0000","last_clean":"2024-09-05T02:01:26.268987+0000","last_became_active":"2024-09-05T02:00:21.412661+0000","last_became_peered":"2024-09-05T02:00:21.412661+0000","last_unstale":"2024-09-05T02:01:26.268987+0000","last_undegraded":"2024-09-05T02:01:26.268987+0000","last_fullsized":"2024-09-05T02:01:26.268987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:51:41.225573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269774+0000","last_change":"2024-09-05T02:00:12.353894+0000","last_active":"2024-09-05T02:01:26.269774+0000","last_peered":"2024-09-05T02:01:26.269774+0000","last_clean":"2024-09-05T02:01:26.269774+0000","last_became_active":"2024-09-05T02:00:12.353483+0000","last_became_peered":"2024-09-05T02:00:12.353483+0000","last_unstale":"2024-09-05T02:01:26.269774+0000","last_undegraded":"2024-09-05T02:01:26.269774+0000","last_fullsized":"2024-09-05T02:01:26.269774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:42.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507738+0000","last_change":"2024-09-05T02:00:14.356441+0000","last_active":"2024-09-05T02:01:26.507738+0000","last_peered":"2024-09-05T02:01:26.507738+0000","last_clean":"2024-09-05T02:01:26.507738+0000","last_became_active":"2024-09-05T02:00:14.356092+0000","last_became_peered":"2024-09-05T02:00:14.356092+0000","last_unstale":"2024-09-05T02:01:26.507738+0000","last_undegraded":"2024-09-05T02:01:26.507738+0000","last_fullsized":"2024-09-05T02:01:26.507738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:33:55.416968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269847+0000","last_change":"2024-09-05T02:00:21.418324+0000","last_active":"2024-09-05T02:01:26.269847+0000","last_peered":"2024-09-05T02:01:26.269847+0000","last_clean":"2024-09-05T02:01:26.269847+0000","last_became_active":"2024-09-05T02:00:21.417934+0000","last_became_peered":"2024-09-05T02:00:21.417934+0000","last_unstale":"2024-09-05T02:01:26.269847+0000","last_undegraded":"2024-09-05T02:01:26.269847+0000","last_fullsized":"2024-09-05T02:01:26.269847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:50.425823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269482+0000","last_change":"2024-09-05T02:00:12.348637+0000","last_active":"2024-09-05T02:01:26.269482+0000","last_peered":"2024-09-05T02:01:26.269482+0000","last_clean":"2024-09-05T02:01:26.269482+0000","last_became_active":"2024-09-05T02:00:12.348287+0000","last_became_peered":"2024-09-05T02:00:12.348287+0000","last_unstale":"2024-09-05T02:01:26.269482+0000","last_undegraded":"2024-09-05T02:01:26.269482+0000","last_fullsized":"2024-09-05T02:01:26.269482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:52:58.368026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268527+0000","last_change":"2024-09-05T02:00:14.368414+0000","last_active":"2024-09-05T02:01:26.268527+0000","last_peered":"2024-09-05T02:01:26.268527+0000","last_clean":"2024-09-05T02:01:26.268527+0000","last_became_active":"2024-09-05T02:00:14.368018+0000","last_became_peered":"2024-09-05T02:00:14.368018+0000","last_unstale":"2024-09-05T02:01:26.268527+0000","last_undegraded":"2024-09-05T02:01:26.268527+0000","last_fullsized":"2024-09-05T02:01:26.268527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:08.246420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268052+0000","last_change":"2024-09-05T02:00:21.416649+0000","last_active":"2024-09-05T02:01:26.268052+0000","last_peered":"2024-09-05T02:01:26.268052+0000","last_clean":"2024-09-05T02:01:26.268052+0000","last_became_active":"2024-09-05T02:00:21.416247+0000","last_became_peered":"2024-09-05T02:00:21.416247+0000","last_unstale":"2024-09-05T02:01:26.268052+0000","last_undegraded":"2024-09-05T02:01:26.268052+0000","last_fullsized":"2024-09-05T02:01:26.268052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:10:00.115655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267664+0000","last_change":"2024-09-05T02:00:14.357719+0000","last_active":"2024-09-05T02:01:26.267664+0000","last_peered":"2024-09-05T02:01:26.267664+0000","last_clean":"2024-09-05T02:01:26.267664+0000","last_became_active":"2024-09-05T02:00:14.357211+0000","last_became_peered":"2024-09-05T02:00:14.357211+0000","last_unstale":"2024-09-05T02:01:26.267664+0000","last_undegraded":"2024-09-05T02:01:26.267664+0000","last_fullsized":"2024-09-05T02:01:26.267664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:27:15.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269584+0000","last_change":"2024-09-05T02:00:12.353893+0000","last_active":"2024-09-05T02:01:26.269584+0000","last_peered":"2024-09-05T02:01:26.269584+0000","last_clean":"2024-09-05T02:01:26.269584+0000","last_became_active":"2024-09-05T02:00:12.353531+0000","last_became_peered":"2024-09-05T02:00:12.353531+0000","last_unstale":"2024-09-05T02:01:26.269584+0000","last_undegraded":"2024-09-05T02:01:26.269584+0000","last_fullsized":"2024-09-05T02:01:26.269584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:58.597446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268248+0000","last_change":"2024-09-05T02:00:21.415203+0000","last_active":"2024-09-05T02:01:26.268248+0000","last_peered":"2024-09-05T02:01:26.268248+0000","last_clean":"2024-09-05T02:01:26.268248+0000","last_became_active":"2024-09-05T02:00:21.414767+0000","last_became_peered":"2024-09-05T02:00:21.414767+0000","last_unstale":"2024-09-05T02:01:26.268248+0000","last_undegraded":"2024-09-05T02:01:26.268248+0000","last_fullsized":"2024-09-05T02:01:26.268248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:01:48.398448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.506997+0000","last_change":"2024-09-05T02:00:14.369417+0000","last_active":"2024-09-05T02:01:26.506997+0000","last_peered":"2024-09-05T02:01:26.506997+0000","last_clean":"2024-09-05T02:01:26.506997+0000","last_became_active":"2024-09-05T02:00:14.369093+0000","last_became_peered":"2024-09-05T02:00:14.369093+0000","last_unstale":"2024-09-05T02:01:26.506997+0000","last_undegraded":"2024-09-05T02:01:26.506997+0000","last_fullsized":"2024-09-05T02:01:26.506997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:01:34.239300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.266047+0000","last_change":"2024-09-05T02:00:12.347945+0000","last_active":"2024-09-05T02:01:26.266047+0000","last_peered":"2024-09-05T02:01:26.266047+0000","last_clean":"2024-09-05T02:01:26.266047+0000","last_became_active":"2024-09-05T02:00:12.347577+0000","last_became_peered":"2024-09-05T02:00:12.347577+0000","last_unstale":"2024-09-05T02:01:26.266047+0000","last_undegraded":"2024-09-05T02:01:26.266047+0000","last_fullsized":"2024-09-05T02:01:26.266047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:53:23.947159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267513+0000","last_change":"2024-09-05T02:00:21.419070+0000","last_active":"2024-09-05T02:01:26.267513+0000","last_peered":"2024-09-05T02:01:26.267513+0000","last_clean":"2024-09-05T02:01:26.267513+0000","last_became_active":"2024-09-05T02:00:21.418315+0000","last_became_peered":"2024-09-05T02:00:21.418315+0000","last_unstale":"2024-09-05T02:01:26.267513+0000","last_undegraded":"2024-09-05T02:01:26.267513+0000","last_fullsized":"2024-09-05T02:01:26.267513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:17.974047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507591+0000","last_change":"2024-09-05T02:00:14.362965+0000","last_active":"2024-09-05T02:01:26.507591+0000","last_peered":"2024-09-05T02:01:26.507591+0000","last_clean":"2024-09-05T02:01:26.507591+0000","last_became_active":"2024-09-05T02:00:14.361842+0000","last_became_peered":"2024-09-05T02:00:14.361842+0000","last_unstale":"2024-09-05T02:01:26.507591+0000","last_undegraded":"2024-09-05T02:01:26.507591+0000","last_fullsized":"2024-09-05T02:01:26.507591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:45:10.565392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507726+0000","last_change":"2024-09-05T02:00:12.349368+0000","last_active":"2024-09-05T02:01:26.507726+0000","last_peered":"2024-09-05T02:01:26.507726+0000","last_clean":"2024-09-05T02:01:26.507726+0000","last_became_active":"2024-09-05T02:00:12.348990+0000","last_became_peered":"2024-09-05T02:00:12.348990+0000","last_unstale":"2024-09-05T02:01:26.507726+0000","last_undegraded":"2024-09-05T02:01:26.507726+0000","last_fullsized":"2024-09-05T02:01:26.507726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:45:04.998316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265703+0000","last_change":"2024-09-05T02:00:21.415467+0000","last_active":"2024-09-05T02:01:26.265703+0000","last_peered":"2024-09-05T02:01:26.265703+0000","last_clean":"2024-09-05T02:01:26.265703+0000","last_became_active":"2024-09-05T02:00:21.415138+0000","last_became_peered":"2024-09-05T02:00:21.415138+0000","last_unstale":"2024-09-05T02:01:26.265703+0000","last_undegraded":"2024-09-05T02:01:26.265703+0000","last_fullsized":"2024-09-05T02:01:26.265703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:51.221383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268684+0000","last_change":"2024-09-05T02:00:14.357961+0000","last_active":"2024-09-05T02:01:26.268684+0000","last_peered":"2024-09-05T02:01:26.268684+0000","last_clean":"2024-09-05T02:01:26.268684+0000","last_became_active":"2024-09-05T02:00:14.357624+0000","last_became_peered":"2024-09-05T02:00:14.357624+0000","last_unstale":"2024-09-05T02:01:26.268684+0000","last_undegraded":"2024-09-05T02:01:26.268684+0000","last_fullsized":"2024-09-05T02:01:26.268684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:01.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508317+0000","last_change":"2024-09-05T02:00:12.783304+0000","last_active":"2024-09-05T02:01:26.508317+0000","last_peered":"2024-09-05T02:01:26.508317+0000","last_clean":"2024-09-05T02:01:26.508317+0000","last_became_active":"2024-09-05T02:00:12.782953+0000","last_became_peered":"2024-09-05T02:00:12.782953+0000","last_unstale":"2024-09-05T02:01:26.508317+0000","last_undegraded":"2024-09-05T02:01:26.508317+0000","last_fullsized":"2024-09-05T02:01:26.508317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:50:59.130705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267510+0000","last_change":"2024-09-05T02:00:21.418357+0000","last_active":"2024-09-05T02:01:26.267510+0000","last_peered":"2024-09-05T02:01:26.267510+0000","last_clean":"2024-09-05T02:01:26.267510+0000","last_became_active":"2024-09-05T02:00:21.417779+0000","last_became_peered":"2024-09-05T02:00:21.417779+0000","last_unstale":"2024-09-05T02:01:26.267510+0000","last_undegraded":"2024-09-05T02:01:26.267510+0000","last_fullsized":"2024-09-05T02:01:26.267510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:37:45.423447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267795+0000","last_change":"2024-09-05T02:00:14.362857+0000","last_active":"2024-09-05T02:01:26.267795+0000","last_peered":"2024-09-05T02:01:26.267795+0000","last_clean":"2024-09-05T02:01:26.267795+0000","last_became_active":"2024-09-05T02:00:14.362556+0000","last_became_peered":"2024-09-05T02:00:14.362556+0000","last_unstale":"2024-09-05T02:01:26.267795+0000","last_undegraded":"2024-09-05T02:01:26.267795+0000","last_fullsized":"2024-09-05T02:01:26.267795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:44:03.413055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268748+0000","last_change":"2024-09-05T02:00:12.512747+0000","last_active":"2024-09-05T02:01:26.268748+0000","last_peered":"2024-09-05T02:01:26.268748+0000","last_clean":"2024-09-05T02:01:26.268748+0000","last_became_active":"2024-09-05T02:00:12.512471+0000","last_became_peered":"2024-09-05T02:00:12.512471+0000","last_unstale":"2024-09-05T02:01:26.268748+0000","last_undegraded":"2024-09-05T02:01:26.268748+0000","last_fullsized":"2024-09-05T02:01:26.268748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:50.235020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265537+0000","last_change":"2024-09-05T02:00:21.413947+0000","last_active":"2024-09-05T02:01:26.265537+0000","last_peered":"2024-09-05T02:01:26.265537+0000","last_clean":"2024-09-05T02:01:26.265537+0000","last_became_active":"2024-09-05T02:00:21.413488+0000","last_became_peered":"2024-09-05T02:00:21.413488+0000","last_unstale":"2024-09-05T02:01:26.265537+0000","last_undegraded":"2024-09-05T02:01:26.265537+0000","last_fullsized":"2024-09-05T02:01:26.265537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:52:16.201325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507067+0000","last_change":"2024-09-05T02:00:14.367009+0000","last_active":"2024-09-05T02:01:26.507067+0000","last_peered":"2024-09-05T02:01:26.507067+0000","last_clean":"2024-09-05T02:01:26.507067+0000","last_became_active":"2024-09-05T02:00:14.366698+0000","last_became_peered":"2024-09-05T02:00:14.366698+0000","last_unstale":"2024-09-05T02:01:26.507067+0000","last_undegraded":"2024-09-05T02:01:26.507067+0000","last_fullsized":"2024-09-05T02:01:26.507067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:37:43.811300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'16","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265846+0000","last_change":"2024-09-05T02:00:12.346918+0000","last_active":"2024-09-05T02:01:26.265846+0000","last_peered":"2024-09-05T02:01:26.265846+0000","last_clean":"2024-09-05T02:01:26.265846+0000","last_became_active":"2024-09-05T02:00:12.346541+0000","last_became_peered":"2024-09-05T02:00:12.346541+0000","last_unstale":"2024-09-05T02:01:26.265846+0000","last_undegraded":"2024-09-05T02:01:26.265846+0000","last_fullsized":"2024-09-05T02:01:26.265846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:50.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507496+0000","last_change":"2024-09-05T02:00:21.413077+0000","last_active":"2024-09-05T02:01:26.507496+0000","last_peered":"2024-09-05T02:01:26.507496+0000","last_clean":"2024-09-05T02:01:26.507496+0000","last_became_active":"2024-09-05T02:00:21.412698+0000","last_became_peered":"2024-09-05T02:00:21.412698+0000","last_unstale":"2024-09-05T02:01:26.507496+0000","last_undegraded":"2024-09-05T02:01:26.507496+0000","last_fullsized":"2024-09-05T02:01:26.507496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:12:18.463614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281275+0000","last_change":"2024-09-05T02:00:14.363072+0000","last_active":"2024-09-05T02:01:26.281275+0000","last_peered":"2024-09-05T02:01:26.281275+0000","last_clean":"2024-09-05T02:01:26.281275+0000","last_became_active":"2024-09-05T02:00:14.361932+0000","last_became_peered":"2024-09-05T02:00:14.361932+0000","last_unstale":"2024-09-05T02:01:26.281275+0000","last_undegraded":"2024-09-05T02:01:26.281275+0000","last_fullsized":"2024-09-05T02:01:26.281275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:20:13.080963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269895+0000","last_change":"2024-09-05T02:00:12.783825+0000","last_active":"2024-09-05T02:01:26.269895+0000","last_peered":"2024-09-05T02:01:26.269895+0000","last_clean":"2024-09-05T02:01:26.269895+0000","last_became_active":"2024-09-05T02:00:12.783298+0000","last_became_peered":"2024-09-05T02:00:12.783298+0000","last_unstale":"2024-09-05T02:01:26.269895+0000","last_undegraded":"2024-09-05T02:01:26.269895+0000","last_fullsized":"2024-09-05T02:01:26.269895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:00:10.986318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508069+0000","last_change":"2024-09-05T02:00:21.416474+0000","last_active":"2024-09-05T02:01:26.508069+0000","last_peered":"2024-09-05T02:01:26.508069+0000","last_clean":"2024-09-05T02:01:26.508069+0000","last_became_active":"2024-09-05T02:00:21.416006+0000","last_became_peered":"2024-09-05T02:00:21.416006+0000","last_unstale":"2024-09-05T02:01:26.508069+0000","last_undegraded":"2024-09-05T02:01:26.508069+0000","last_fullsized":"2024-09-05T02:01:26.508069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:00:18.587895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265953+0000","last_change":"2024-09-05T02:00:14.362458+0000","last_active":"2024-09-05T02:01:26.265953+0000","last_peered":"2024-09-05T02:01:26.265953+0000","last_clean":"2024-09-05T02:01:26.265953+0000","last_became_active":"2024-09-05T02:00:14.362080+0000","last_became_peered":"2024-09-05T02:00:14.362080+0000","last_unstale":"2024-09-05T02:01:26.265953+0000","last_undegraded":"2024-09-05T02:01:26.265953+0000","last_fullsized":"2024-09-05T02:01:26.265953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:47:49.498468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265949+0000","last_change":"2024-09-05T02:00:12.783445+0000","last_active":"2024-09-05T02:01:26.265949+0000","last_peered":"2024-09-05T02:01:26.265949+0000","last_clean":"2024-09-05T02:01:26.265949+0000","last_became_active":"2024-09-05T02:00:12.783131+0000","last_became_peered":"2024-09-05T02:00:12.783131+0000","last_unstale":"2024-09-05T02:01:26.265949+0000","last_undegraded":"2024-09-05T02:01:26.265949+0000","last_fullsized":"2024-09-05T02:01:26.265949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:21:57.743629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268951+0000","last_change":"2024-09-05T02:00:21.418568+0000","last_active":"2024-09-05T02:01:26.268951+0000","last_peered":"2024-09-05T02:01:26.268951+0000","last_clean":"2024-09-05T02:01:26.268951+0000","last_became_active":"2024-09-05T02:00:21.418152+0000","last_became_peered":"2024-09-05T02:00:21.418152+0000","last_unstale":"2024-09-05T02:01:26.268951+0000","last_undegraded":"2024-09-05T02:01:26.268951+0000","last_fullsized":"2024-09-05T02:01:26.268951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:50:17.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265914+0000","last_change":"2024-09-05T02:00:14.364556+0000","last_active":"2024-09-05T02:01:26.265914+0000","last_peered":"2024-09-05T02:01:26.265914+0000","last_clean":"2024-09-05T02:01:26.265914+0000","last_became_active":"2024-09-05T02:00:14.364285+0000","last_became_peered":"2024-09-05T02:00:14.364285+0000","last_unstale":"2024-09-05T02:01:26.265914+0000","last_undegraded":"2024-09-05T02:01:26.265914+0000","last_fullsized":"2024-09-05T02:01:26.265914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:24:54.901510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:50.309622+0000","last_change":"2024-09-05T02:00:12.347993+0000","last_active":"2024-09-05T02:01:50.309622+0000","last_peered":"2024-09-05T02:01:50.309622+0000","last_clean":"2024-09-05T02:01:50.309622+0000","last_became_active":"2024-09-05T02:00:12.347656+0000","last_became_peered":"2024-09-05T02:00:12.347656+0000","last_unstale":"2024-09-05T02:01:50.309622+0000","last_undegraded":"2024-09-05T02:01:50.309622+0000","last_fullsized":"2024-09-05T02:01:50.309622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:31:23.820379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281122+0000","last_change":"2024-09-05T02:00:21.410786+0000","last_active":"2024-09-05T02:01:26.281122+0000","last_peered":"2024-09-05T02:01:26.281122+0000","last_clean":"2024-09-05T02:01:26.281122+0000","last_became_active":"2024-09-05T02:00:21.410440+0000","last_became_peered":"2024-09-05T02:00:21.410440+0000","last_unstale":"2024-09-05T02:01:26.281122+0000","last_undegraded":"2024-09-05T02:01:26.281122+0000","last_fullsized":"2024-09-05T02:01:26.281122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:22:14.644413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.282147+0000","last_change":"2024-09-05T02:01:26.282147+0000","last_active":"2024-09-05T02:01:26.282147+0000","last_peered":"2024-09-05T02:01:26.282147+0000","last_clean":"2024-09-05T02:01:26.282147+0000","last_became_active":"2024-09-05T02:01:26.281792+0000","last_became_peered":"2024-09-05T02:01:26.281792+0000","last_unstale":"2024-09-05T02:01:26.282147+0000","last_undegraded":"2024-09-05T02:01:26.282147+0000","last_fullsized":"2024-09-05T02:01:26.282147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:10:43.403518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:46.303986+0000","last_change":"2024-09-05T02:00:12.783358+0000","last_active":"2024-09-05T02:01:46.303986+0000","last_peered":"2024-09-05T02:01:46.303986+0000","last_clean":"2024-09-05T02:01:46.303986+0000","last_became_active":"2024-09-05T02:00:12.782950+0000","last_became_peered":"2024-09-05T02:00:12.782950+0000","last_unstale":"2024-09-05T02:01:46.303986+0000","last_undegraded":"2024-09-05T02:01:46.303986+0000","last_fullsized":"2024-09-05T02:01:46.303986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:17:38.823019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281218+0000","last_change":"2024-09-05T02:00:21.416389+0000","last_active":"2024-09-05T02:01:26.281218+0000","last_peered":"2024-09-05T02:01:26.281218+0000","last_clean":"2024-09-05T02:01:26.281218+0000","last_became_active":"2024-09-05T02:00:21.416065+0000","last_became_peered":"2024-09-05T02:00:21.416065+0000","last_unstale":"2024-09-05T02:01:26.281218+0000","last_undegraded":"2024-09-05T02:01:26.281218+0000","last_fullsized":"2024-09-05T02:01:26.281218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:15.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.280762+0000","last_change":"2024-09-05T02:00:14.361404+0000","last_active":"2024-09-05T02:01:26.280762+0000","last_peered":"2024-09-05T02:01:26.280762+0000","last_clean":"2024-09-05T02:01:26.280762+0000","last_became_active":"2024-09-05T02:00:14.361031+0000","last_became_peered":"2024-09-05T02:00:14.361031+0000","last_unstale":"2024-09-05T02:01:26.280762+0000","last_undegraded":"2024-09-05T02:01:26.280762+0000","last_fullsized":"2024-09-05T02:01:26.280762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:40.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267876+0000","last_change":"2024-09-05T02:00:12.351656+0000","last_active":"2024-09-05T02:01:26.267876+0000","last_peered":"2024-09-05T02:01:26.267876+0000","last_clean":"2024-09-05T02:01:26.267876+0000","last_became_active":"2024-09-05T02:00:12.351310+0000","last_became_peered":"2024-09-05T02:00:12.351310+0000","last_unstale":"2024-09-05T02:01:26.267876+0000","last_undegraded":"2024-09-05T02:01:26.267876+0000","last_fullsized":"2024-09-05T02:01:26.267876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:55:27.949471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.510839+0000","last_change":"2024-09-05T02:01:26.510839+0000","last_active":"2024-09-05T02:01:26.510839+0000","last_peered":"2024-09-05T02:01:26.510839+0000","last_clean":"2024-09-05T02:01:26.510839+0000","last_became_active":"2024-09-05T02:01:26.509959+0000","last_became_peered":"2024-09-05T02:01:26.509959+0000","last_unstale":"2024-09-05T02:01:26.510839+0000","last_undegraded":"2024-09-05T02:01:26.510839+0000","last_fullsized":"2024-09-05T02:01:26.510839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:57.240190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507027+0000","last_change":"2024-09-05T02:00:14.359356+0000","last_active":"2024-09-05T02:01:26.507027+0000","last_peered":"2024-09-05T02:01:26.507027+0000","last_clean":"2024-09-05T02:01:26.507027+0000","last_became_active":"2024-09-05T02:00:14.359078+0000","last_became_peered":"2024-09-05T02:00:14.359078+0000","last_unstale":"2024-09-05T02:01:26.507027+0000","last_undegraded":"2024-09-05T02:01:26.507027+0000","last_fullsized":"2024-09-05T02:01:26.507027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:40:24.219962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.270304+0000","last_change":"2024-09-05T02:00:12.349749+0000","last_active":"2024-09-05T02:01:26.270304+0000","last_peered":"2024-09-05T02:01:26.270304+0000","last_clean":"2024-09-05T02:01:26.270304+0000","last_became_active":"2024-09-05T02:00:12.349247+0000","last_became_peered":"2024-09-05T02:00:12.349247+0000","last_unstale":"2024-09-05T02:01:26.270304+0000","last_undegraded":"2024-09-05T02:01:26.270304+0000","last_fullsized":"2024-09-05T02:01:26.270304+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:04:26.361209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268031+0000","last_change":"2024-09-05T01:58:57.552989+0000","last_active":"2024-09-05T02:01:26.268031+0000","last_peered":"2024-09-05T02:01:26.268031+0000","last_clean":"2024-09-05T02:01:26.268031+0000","last_became_active":"2024-09-05T01:58:57.552511+0000","last_became_peered":"2024-09-05T01:58:57.552511+0000","last_unstale":"2024-09-05T02:01:26.268031+0000","last_undegraded":"2024-09-05T02:01:26.268031+0000","last_fullsized":"2024-09-05T02:01:26.268031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_clean_scrub_stamp":"2024-09-05T01:55:35.678445+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:53.711210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268473+0000","last_change":"2024-09-05T02:00:21.421456+0000","last_active":"2024-09-05T02:01:26.268473+0000","last_peered":"2024-09-05T02:01:26.268473+0000","last_clean":"2024-09-05T02:01:26.268473+0000","last_became_active":"2024-09-05T02:00:21.421074+0000","last_became_peered":"2024-09-05T02:00:21.421074+0000","last_unstale":"2024-09-05T02:01:26.268473+0000","last_undegraded":"2024-09-05T02:01:26.268473+0000","last_fullsized":"2024-09-05T02:01:26.268473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:46:00.660079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268706+0000","last_change":"2024-09-05T02:00:14.357638+0000","last_active":"2024-09-05T02:01:26.268706+0000","last_peered":"2024-09-05T02:01:26.268706+0000","last_clean":"2024-09-05T02:01:26.268706+0000","last_became_active":"2024-09-05T02:00:14.357125+0000","last_became_peered":"2024-09-05T02:00:14.357125+0000","last_unstale":"2024-09-05T02:01:26.268706+0000","last_undegraded":"2024-09-05T02:01:26.268706+0000","last_fullsized":"2024-09-05T02:01:26.268706+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:02:04.343235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:48.307918+0000","last_change":"2024-09-05T02:00:12.783627+0000","last_active":"2024-09-05T02:01:48.307918+0000","last_peered":"2024-09-05T02:01:48.307918+0000","last_clean":"2024-09-05T02:01:48.307918+0000","last_became_active":"2024-09-05T02:00:12.783112+0000","last_became_peered":"2024-09-05T02:00:12.783112+0000","last_unstale":"2024-09-05T02:01:48.307918+0000","last_undegraded":"2024-09-05T02:01:48.307918+0000","last_fullsized":"2024-09-05T02:01:48.307918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:39:45.190311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508601+0000","last_change":"2024-09-05T02:00:21.413069+0000","last_active":"2024-09-05T02:01:26.508601+0000","last_peered":"2024-09-05T02:01:26.508601+0000","last_clean":"2024-09-05T02:01:26.508601+0000","last_became_active":"2024-09-05T02:00:21.412687+0000","last_became_peered":"2024-09-05T02:00:21.412687+0000","last_unstale":"2024-09-05T02:01:26.508601+0000","last_undegraded":"2024-09-05T02:01:26.508601+0000","last_fullsized":"2024-09-05T02:01:26.508601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:28.883071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268278+0000","last_change":"2024-09-05T02:00:14.363197+0000","last_active":"2024-09-05T02:01:26.268278+0000","last_peered":"2024-09-05T02:01:26.268278+0000","last_clean":"2024-09-05T02:01:26.268278+0000","last_became_active":"2024-09-05T02:00:14.362854+0000","last_became_peered":"2024-09-05T02:00:14.362854+0000","last_unstale":"2024-09-05T02:01:26.268278+0000","last_undegraded":"2024-09-05T02:01:26.268278+0000","last_fullsized":"2024-09-05T02:01:26.268278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:29:28.770604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269052+0000","last_change":"2024-09-05T02:00:12.348715+0000","last_active":"2024-09-05T02:01:26.269052+0000","last_peered":"2024-09-05T02:01:26.269052+0000","last_clean":"2024-09-05T02:01:26.269052+0000","last_became_active":"2024-09-05T02:00:12.348322+0000","last_became_peered":"2024-09-05T02:00:12.348322+0000","last_unstale":"2024-09-05T02:01:26.269052+0000","last_undegraded":"2024-09-05T02:01:26.269052+0000","last_fullsized":"2024-09-05T02:01:26.269052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:33:55.551080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.509548+0000","last_change":"2024-09-05T02:01:26.509548+0000","last_active":"2024-09-05T02:01:26.509548+0000","last_peered":"2024-09-05T02:01:26.509548+0000","last_clean":"2024-09-05T02:01:26.509548+0000","last_became_active":"2024-09-05T02:01:26.508993+0000","last_became_peered":"2024-09-05T02:01:26.508993+0000","last_unstale":"2024-09-05T02:01:26.509548+0000","last_undegraded":"2024-09-05T02:01:26.509548+0000","last_fullsized":"2024-09-05T02:01:26.509548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:46.109685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268767+0000","last_change":"2024-09-05T02:00:14.355969+0000","last_active":"2024-09-05T02:01:26.268767+0000","last_peered":"2024-09-05T02:01:26.268767+0000","last_clean":"2024-09-05T02:01:26.268767+0000","last_became_active":"2024-09-05T02:00:14.355581+0000","last_became_peered":"2024-09-05T02:00:14.355581+0000","last_unstale":"2024-09-05T02:01:26.268767+0000","last_undegraded":"2024-09-05T02:01:26.268767+0000","last_fullsized":"2024-09-05T02:01:26.268767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:55:22.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281299+0000","last_change":"2024-09-05T02:00:12.348148+0000","last_active":"2024-09-05T02:01:26.281299+0000","last_peered":"2024-09-05T02:01:26.281299+0000","last_clean":"2024-09-05T02:01:26.281299+0000","last_became_active":"2024-09-05T02:00:12.347857+0000","last_became_peered":"2024-09-05T02:00:12.347857+0000","last_unstale":"2024-09-05T02:01:26.281299+0000","last_undegraded":"2024-09-05T02:01:26.281299+0000","last_fullsized":"2024-09-05T02:01:26.281299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:22:21.368082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268247+0000","last_change":"2024-09-05T02:00:21.417476+0000","last_active":"2024-09-05T02:01:26.268247+0000","last_peered":"2024-09-05T02:01:26.268247+0000","last_clean":"2024-09-05T02:01:26.268247+0000","last_became_active":"2024-09-05T02:00:21.417036+0000","last_became_peered":"2024-09-05T02:00:21.417036+0000","last_unstale":"2024-09-05T02:01:26.268247+0000","last_undegraded":"2024-09-05T02:01:26.268247+0000","last_fullsized":"2024-09-05T02:01:26.268247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:15:22.510948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507726+0000","last_change":"2024-09-05T02:00:14.358542+0000","last_active":"2024-09-05T02:01:26.507726+0000","last_peered":"2024-09-05T02:01:26.507726+0000","last_clean":"2024-09-05T02:01:26.507726+0000","last_became_active":"2024-09-05T02:00:14.358228+0000","last_became_peered":"2024-09-05T02:00:14.358228+0000","last_unstale":"2024-09-05T02:01:26.507726+0000","last_undegraded":"2024-09-05T02:01:26.507726+0000","last_fullsized":"2024-09-05T02:01:26.507726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:01.627453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.270241+0000","last_change":"2024-09-05T02:00:12.345887+0000","last_active":"2024-09-05T02:01:26.270241+0000","last_peered":"2024-09-05T02:01:26.270241+0000","last_clean":"2024-09-05T02:01:26.270241+0000","last_became_active":"2024-09-05T02:00:12.345491+0000","last_became_peered":"2024-09-05T02:00:12.345491+0000","last_unstale":"2024-09-05T02:01:26.270241+0000","last_undegraded":"2024-09-05T02:01:26.270241+0000","last_fullsized":"2024-09-05T02:01:26.270241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:10:32.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268661+0000","last_change":"2024-09-05T02:00:21.418210+0000","last_active":"2024-09-05T02:01:26.268661+0000","last_peered":"2024-09-05T02:01:26.268661+0000","last_clean":"2024-09-05T02:01:26.268661+0000","last_became_active":"2024-09-05T02:00:21.417769+0000","last_became_peered":"2024-09-05T02:00:21.417769+0000","last_unstale":"2024-09-05T02:01:26.268661+0000","last_undegraded":"2024-09-05T02:01:26.268661+0000","last_fullsized":"2024-09-05T02:01:26.268661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:05.050499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269951+0000","last_change":"2024-09-05T02:00:14.367555+0000","last_active":"2024-09-05T02:01:26.269951+0000","last_peered":"2024-09-05T02:01:26.269951+0000","last_clean":"2024-09-05T02:01:26.269951+0000","last_became_active":"2024-09-05T02:00:14.367222+0000","last_became_peered":"2024-09-05T02:00:14.367222+0000","last_unstale":"2024-09-05T02:01:26.269951+0000","last_undegraded":"2024-09-05T02:01:26.269951+0000","last_fullsized":"2024-09-05T02:01:26.269951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:04:26.172141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.282875+0000","last_change":"2024-09-05T02:01:26.282875+0000","last_active":"2024-09-05T02:01:26.282875+0000","last_peered":"2024-09-05T02:01:26.282875+0000","last_clean":"2024-09-05T02:01:26.282875+0000","last_became_active":"2024-09-05T02:01:26.271901+0000","last_became_peered":"2024-09-05T02:01:26.271901+0000","last_unstale":"2024-09-05T02:01:26.282875+0000","last_undegraded":"2024-09-05T02:01:26.282875+0000","last_fullsized":"2024-09-05T02:01:26.282875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:29:45.232870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281297+0000","last_change":"2024-09-05T02:00:21.412374+0000","last_active":"2024-09-05T02:01:26.281297+0000","last_peered":"2024-09-05T02:01:26.281297+0000","last_clean":"2024-09-05T02:01:26.281297+0000","last_became_active":"2024-09-05T02:00:21.411926+0000","last_became_peered":"2024-09-05T02:00:21.411926+0000","last_unstale":"2024-09-05T02:01:26.281297+0000","last_undegraded":"2024-09-05T02:01:26.281297+0000","last_fullsized":"2024-09-05T02:01:26.281297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:38:29.657795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269814+0000","last_change":"2024-09-05T02:01:26.269814+0000","last_active":"2024-09-05T02:01:26.269814+0000","last_peered":"2024-09-05T02:01:26.269814+0000","last_clean":"2024-09-05T02:01:26.269814+0000","last_became_active":"2024-09-05T02:01:26.269320+0000","last_became_peered":"2024-09-05T02:01:26.269320+0000","last_unstale":"2024-09-05T02:01:26.269814+0000","last_undegraded":"2024-09-05T02:01:26.269814+0000","last_fullsized":"2024-09-05T02:01:26.269814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:04:21.161986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:47.305810+0000","last_change":"2024-09-05T02:00:12.349175+0000","last_active":"2024-09-05T02:01:47.305810+0000","last_peered":"2024-09-05T02:01:47.305810+0000","last_clean":"2024-09-05T02:01:47.305810+0000","last_became_active":"2024-09-05T02:00:12.348647+0000","last_became_peered":"2024-09-05T02:00:12.348647+0000","last_unstale":"2024-09-05T02:01:47.305810+0000","last_undegraded":"2024-09-05T02:01:47.305810+0000","last_fullsized":"2024-09-05T02:01:47.305810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:38:13.363462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265680+0000","last_change":"2024-09-05T02:00:21.414252+0000","last_active":"2024-09-05T02:01:26.265680+0000","last_peered":"2024-09-05T02:01:26.265680+0000","last_clean":"2024-09-05T02:01:26.265680+0000","last_became_active":"2024-09-05T02:00:21.412483+0000","last_became_peered":"2024-09-05T02:00:21.412483+0000","last_unstale":"2024-09-05T02:01:26.265680+0000","last_undegraded":"2024-09-05T02:01:26.265680+0000","last_fullsized":"2024-09-05T02:01:26.265680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:58.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268173+0000","last_change":"2024-09-05T02:00:14.361018+0000","last_active":"2024-09-05T02:01:26.268173+0000","last_peered":"2024-09-05T02:01:26.268173+0000","last_clean":"2024-09-05T02:01:26.268173+0000","last_became_active":"2024-09-05T02:00:14.360591+0000","last_became_peered":"2024-09-05T02:00:14.360591+0000","last_unstale":"2024-09-05T02:01:26.268173+0000","last_undegraded":"2024-09-05T02:01:26.268173+0000","last_fullsized":"2024-09-05T02:01:26.268173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:49.175681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268593+0000","last_change":"2024-09-05T02:00:12.353257+0000","last_active":"2024-09-05T02:01:26.268593+0000","last_peered":"2024-09-05T02:01:26.268593+0000","last_clean":"2024-09-05T02:01:26.268593+0000","last_became_active":"2024-09-05T02:00:12.352906+0000","last_became_peered":"2024-09-05T02:00:12.352906+0000","last_unstale":"2024-09-05T02:01:26.268593+0000","last_undegraded":"2024-09-05T02:01:26.268593+0000","last_fullsized":"2024-09-05T02:01:26.268593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:53:52.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508032+0000","last_change":"2024-09-05T02:00:21.410346+0000","last_active":"2024-09-05T02:01:26.508032+0000","last_peered":"2024-09-05T02:01:26.508032+0000","last_clean":"2024-09-05T02:01:26.508032+0000","last_became_active":"2024-09-05T02:00:21.409946+0000","last_became_peered":"2024-09-05T02:00:21.409946+0000","last_unstale":"2024-09-05T02:01:26.508032+0000","last_undegraded":"2024-09-05T02:01:26.508032+0000","last_fullsized":"2024-09-05T02:01:26.508032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:56.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.280907+0000","last_change":"2024-09-05T02:00:14.360265+0000","last_active":"2024-09-05T02:01:26.280907+0000","last_peered":"2024-09-05T02:01:26.280907+0000","last_clean":"2024-09-05T02:01:26.280907+0000","last_became_active":"2024-09-05T02:00:14.359958+0000","last_became_peered":"2024-09-05T02:00:14.359958+0000","last_unstale":"2024-09-05T02:01:26.280907+0000","last_undegraded":"2024-09-05T02:01:26.280907+0000","last_fullsized":"2024-09-05T02:01:26.280907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:15:45.211207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265890+0000","last_change":"2024-09-05T02:00:12.349443+0000","last_active":"2024-09-05T02:01:26.265890+0000","last_peered":"2024-09-05T02:01:26.265890+0000","last_clean":"2024-09-05T02:01:26.265890+0000","last_became_active":"2024-09-05T02:00:12.348988+0000","last_became_peered":"2024-09-05T02:00:12.348988+0000","last_unstale":"2024-09-05T02:01:26.265890+0000","last_undegraded":"2024-09-05T02:01:26.265890+0000","last_fullsized":"2024-09-05T02:01:26.265890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:14:21.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268391+0000","last_change":"2024-09-05T02:00:21.417662+0000","last_active":"2024-09-05T02:01:26.268391+0000","last_peered":"2024-09-05T02:01:26.268391+0000","last_clean":"2024-09-05T02:01:26.268391+0000","last_became_active":"2024-09-05T02:00:21.417238+0000","last_became_peered":"2024-09-05T02:00:21.417238+0000","last_unstale":"2024-09-05T02:01:26.268391+0000","last_undegraded":"2024-09-05T02:01:26.268391+0000","last_fullsized":"2024-09-05T02:01:26.268391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:05:59.304480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269560+0000","last_change":"2024-09-05T02:00:14.355928+0000","last_active":"2024-09-05T02:01:26.269560+0000","last_peered":"2024-09-05T02:01:26.269560+0000","last_clean":"2024-09-05T02:01:26.269560+0000","last_became_active":"2024-09-05T02:00:14.355576+0000","last_became_peered":"2024-09-05T02:00:14.355576+0000","last_unstale":"2024-09-05T02:01:26.269560+0000","last_undegraded":"2024-09-05T02:01:26.269560+0000","last_fullsized":"2024-09-05T02:01:26.269560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:22:50.095434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'34","reported_seq":177,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:48.307917+0000","last_change":"2024-09-05T02:00:12.352198+0000","last_active":"2024-09-05T02:01:48.307917+0000","last_peered":"2024-09-05T02:01:48.307917+0000","last_clean":"2024-09-05T02:01:48.307917+0000","last_became_active":"2024-09-05T02:00:12.351806+0000","last_became_peered":"2024-09-05T02:00:12.351806+0000","last_unstale":"2024-09-05T02:01:48.307917+0000","last_undegraded":"2024-09-05T02:01:48.307917+0000","last_fullsized":"2024-09-05T02:01:48.307917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.178160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":115,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.509575+0000","last_change":"2024-09-05T02:01:26.509575+0000","last_active":"2024-09-05T02:01:26.509575+0000","last_peered":"2024-09-05T02:01:26.509575+0000","last_clean":"2024-09-05T02:01:26.509575+0000","last_became_active":"2024-09-05T02:01:26.509108+0000","last_became_peered":"2024-09-05T02:01:26.509108+0000","last_unstale":"2024-09-05T02:01:26.509575+0000","last_undegraded":"2024-09-05T02:01:26.509575+0000","last_fullsized":"2024-09-05T02:01:26.509575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:55.503181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507570+0000","last_change":"2024-09-05T02:00:14.365372+0000","last_active":"2024-09-05T02:01:26.507570+0000","last_peered":"2024-09-05T02:01:26.507570+0000","last_clean":"2024-09-05T02:01:26.507570+0000","last_became_active":"2024-09-05T02:00:14.365064+0000","last_became_peered":"2024-09-05T02:00:14.365064+0000","last_unstale":"2024-09-05T02:01:26.507570+0000","last_undegraded":"2024-09-05T02:01:26.507570+0000","last_fullsized":"2024-09-05T02:01:26.507570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:02:29.378126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269261+0000","last_change":"2024-09-05T02:00:12.783487+0000","last_active":"2024-09-05T02:01:26.269261+0000","last_peered":"2024-09-05T02:01:26.269261+0000","last_clean":"2024-09-05T02:01:26.269261+0000","last_became_active":"2024-09-05T02:00:12.783087+0000","last_became_peered":"2024-09-05T02:00:12.783087+0000","last_unstale":"2024-09-05T02:01:26.269261+0000","last_undegraded":"2024-09-05T02:01:26.269261+0000","last_fullsized":"2024-09-05T02:01:26.269261+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:08.574845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.510122+0000","last_change":"2024-09-05T02:01:26.510122+0000","last_active":"2024-09-05T02:01:26.510122+0000","last_peered":"2024-09-05T02:01:26.510122+0000","last_clean":"2024-09-05T02:01:26.510122+0000","last_became_active":"2024-09-05T02:01:26.509427+0000","last_became_peered":"2024-09-05T02:01:26.509427+0000","last_unstale":"2024-09-05T02:01:26.510122+0000","last_undegraded":"2024-09-05T02:01:26.510122+0000","last_fullsized":"2024-09-05T02:01:26.510122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:50.139855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.272703+0000","last_change":"2024-09-05T02:01:26.272703+0000","last_active":"2024-09-05T02:01:26.272703+0000","last_peered":"2024-09-05T02:01:26.272703+0000","last_clean":"2024-09-05T02:01:26.272703+0000","last_became_active":"2024-09-05T02:01:26.272440+0000","last_became_peered":"2024-09-05T02:01:26.272440+0000","last_unstale":"2024-09-05T02:01:26.272703+0000","last_undegraded":"2024-09-05T02:01:26.272703+0000","last_fullsized":"2024-09-05T02:01:26.272703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:29.855027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268556+0000","last_change":"2024-09-05T02:00:12.349362+0000","last_active":"2024-09-05T02:01:26.268556+0000","last_peered":"2024-09-05T02:01:26.268556+0000","last_clean":"2024-09-05T02:01:26.268556+0000","last_became_active":"2024-09-05T02:00:12.349016+0000","last_became_peered":"2024-09-05T02:00:12.349016+0000","last_unstale":"2024-09-05T02:01:26.268556+0000","last_undegraded":"2024-09-05T02:01:26.268556+0000","last_fullsized":"2024-09-05T02:01:26.268556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:37.603717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265157+0000","last_change":"2024-09-05T02:00:21.412942+0000","last_active":"2024-09-05T02:01:26.265157+0000","last_peered":"2024-09-05T02:01:26.265157+0000","last_clean":"2024-09-05T02:01:26.265157+0000","last_became_active":"2024-09-05T02:00:21.412057+0000","last_became_peered":"2024-09-05T02:00:21.412057+0000","last_unstale":"2024-09-05T02:01:26.265157+0000","last_undegraded":"2024-09-05T02:01:26.265157+0000","last_fullsized":"2024-09-05T02:01:26.265157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:59.872818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507944+0000","last_change":"2024-09-05T02:00:14.356636+0000","last_active":"2024-09-05T02:01:26.507944+0000","last_peered":"2024-09-05T02:01:26.507944+0000","last_clean":"2024-09-05T02:01:26.507944+0000","last_became_active":"2024-09-05T02:00:14.356321+0000","last_became_peered":"2024-09-05T02:00:14.356321+0000","last_unstale":"2024-09-05T02:01:26.507944+0000","last_undegraded":"2024-09-05T02:01:26.507944+0000","last_fullsized":"2024-09-05T02:01:26.507944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:02.112646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":138,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:48.307387+0000","last_change":"2024-09-05T02:00:12.353078+0000","last_active":"2024-09-05T02:01:48.307387+0000","last_peered":"2024-09-05T02:01:48.307387+0000","last_clean":"2024-09-05T02:01:48.307387+0000","last_became_active":"2024-09-05T02:00:12.352707+0000","last_became_peered":"2024-09-05T02:00:12.352707+0000","last_unstale":"2024-09-05T02:01:48.307387+0000","last_undegraded":"2024-09-05T02:01:48.307387+0000","last_fullsized":"2024-09-05T02:01:48.307387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:28:29.934304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507633+0000","last_change":"2024-09-05T02:00:21.421362+0000","last_active":"2024-09-05T02:01:26.507633+0000","last_peered":"2024-09-05T02:01:26.507633+0000","last_clean":"2024-09-05T02:01:26.507633+0000","last_became_active":"2024-09-05T02:00:21.420998+0000","last_became_peered":"2024-09-05T02:00:21.420998+0000","last_unstale":"2024-09-05T02:01:26.507633+0000","last_undegraded":"2024-09-05T02:01:26.507633+0000","last_fullsized":"2024-09-05T02:01:26.507633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:26:14.105489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267672+0000","last_change":"2024-09-05T02:00:21.416703+0000","last_active":"2024-09-05T02:01:26.267672+0000","last_peered":"2024-09-05T02:01:26.267672+0000","last_clean":"2024-09-05T02:01:26.267672+0000","last_became_active":"2024-09-05T02:00:21.415913+0000","last_became_peered":"2024-09-05T02:00:21.415913+0000","last_unstale":"2024-09-05T02:01:26.267672+0000","last_undegraded":"2024-09-05T02:01:26.267672+0000","last_fullsized":"2024-09-05T02:01:26.267672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:56.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265232+0000","last_change":"2024-09-05T02:00:14.355658+0000","last_active":"2024-09-05T02:01:26.265232+0000","last_peered":"2024-09-05T02:01:26.265232+0000","last_clean":"2024-09-05T02:01:26.265232+0000","last_became_active":"2024-09-05T02:00:14.355087+0000","last_became_peered":"2024-09-05T02:00:14.355087+0000","last_unstale":"2024-09-05T02:01:26.265232+0000","last_undegraded":"2024-09-05T02:01:26.265232+0000","last_fullsized":"2024-09-05T02:01:26.265232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:58:08.369027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269514+0000","last_change":"2024-09-05T02:00:12.348171+0000","last_active":"2024-09-05T02:01:26.269514+0000","last_peered":"2024-09-05T02:01:26.269514+0000","last_clean":"2024-09-05T02:01:26.269514+0000","last_became_active":"2024-09-05T02:00:12.347760+0000","last_became_peered":"2024-09-05T02:00:12.347760+0000","last_unstale":"2024-09-05T02:01:26.269514+0000","last_undegraded":"2024-09-05T02:01:26.269514+0000","last_fullsized":"2024-09-05T02:01:26.269514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:03:43.748458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508807+0000","last_change":"2024-09-05T02:00:21.413015+0000","last_active":"2024-09-05T02:01:26.508807+0000","last_peered":"2024-09-05T02:01:26.508807+0000","last_clean":"2024-09-05T02:01:26.508807+0000","last_became_active":"2024-09-05T02:00:21.412631+0000","last_became_peered":"2024-09-05T02:00:21.412631+0000","last_unstale":"2024-09-05T02:01:26.508807+0000","last_undegraded":"2024-09-05T02:01:26.508807+0000","last_fullsized":"2024-09-05T02:01:26.508807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:35.094273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.506938+0000","last_change":"2024-09-05T02:00:14.354552+0000","last_active":"2024-09-05T02:01:26.506938+0000","last_peered":"2024-09-05T02:01:26.506938+0000","last_clean":"2024-09-05T02:01:26.506938+0000","last_became_active":"2024-09-05T02:00:14.354150+0000","last_became_peered":"2024-09-05T02:00:14.354150+0000","last_unstale":"2024-09-05T02:01:26.506938+0000","last_undegraded":"2024-09-05T02:01:26.506938+0000","last_fullsized":"2024-09-05T02:01:26.506938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:16:19.836148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.280777+0000","last_change":"2024-09-05T02:00:12.342608+0000","last_active":"2024-09-05T02:01:26.280777+0000","last_peered":"2024-09-05T02:01:26.280777+0000","last_clean":"2024-09-05T02:01:26.280777+0000","last_became_active":"2024-09-05T02:00:12.342235+0000","last_became_peered":"2024-09-05T02:00:12.342235+0000","last_unstale":"2024-09-05T02:01:26.280777+0000","last_undegraded":"2024-09-05T02:01:26.280777+0000","last_fullsized":"2024-09-05T02:01:26.280777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:54.568963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265875+0000","last_change":"2024-09-05T02:00:12.352247+0000","last_active":"2024-09-05T02:01:26.265875+0000","last_peered":"2024-09-05T02:01:26.265875+0000","last_clean":"2024-09-05T02:01:26.265875+0000","last_became_active":"2024-09-05T02:00:12.351897+0000","last_became_peered":"2024-09-05T02:00:12.351897+0000","last_unstale":"2024-09-05T02:01:26.265875+0000","last_undegraded":"2024-09-05T02:01:26.265875+0000","last_fullsized":"2024-09-05T02:01:26.265875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:48.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268681+0000","last_change":"2024-09-05T02:00:14.366415+0000","last_active":"2024-09-05T02:01:26.268681+0000","last_peered":"2024-09-05T02:01:26.268681+0000","last_clean":"2024-09-05T02:01:26.268681+0000","last_became_active":"2024-09-05T02:00:14.366103+0000","last_became_peered":"2024-09-05T02:00:14.366103+0000","last_unstale":"2024-09-05T02:01:26.268681+0000","last_undegraded":"2024-09-05T02:01:26.268681+0000","last_fullsized":"2024-09-05T02:01:26.268681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:22:21.585867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265920+0000","last_change":"2024-09-05T02:00:21.416879+0000","last_active":"2024-09-05T02:01:26.265920+0000","last_peered":"2024-09-05T02:01:26.265920+0000","last_clean":"2024-09-05T02:01:26.265920+0000","last_became_active":"2024-09-05T02:00:21.415887+0000","last_became_peered":"2024-09-05T02:00:21.415887+0000","last_unstale":"2024-09-05T02:01:26.265920+0000","last_undegraded":"2024-09-05T02:01:26.265920+0000","last_fullsized":"2024-09-05T02:01:26.265920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:58.106529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:48.308280+0000","last_change":"2024-09-05T02:00:12.354055+0000","last_active":"2024-09-05T02:01:48.308280+0000","last_peered":"2024-09-05T02:01:48.308280+0000","last_clean":"2024-09-05T02:01:48.308280+0000","last_became_active":"2024-09-05T02:00:12.353754+0000","last_became_peered":"2024-09-05T02:00:12.353754+0000","last_unstale":"2024-09-05T02:01:48.308280+0000","last_undegraded":"2024-09-05T02:01:48.308280+0000","last_fullsized":"2024-09-05T02:01:48.308280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:17.671079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269072+0000","last_change":"2024-09-05T02:00:14.358646+0000","last_active":"2024-09-05T02:01:26.269072+0000","last_peered":"2024-09-05T02:01:26.269072+0000","last_clean":"2024-09-05T02:01:26.269072+0000","last_became_active":"2024-09-05T02:00:14.358337+0000","last_became_peered":"2024-09-05T02:00:14.358337+0000","last_unstale":"2024-09-05T02:01:26.269072+0000","last_undegraded":"2024-09-05T02:01:26.269072+0000","last_fullsized":"2024-09-05T02:01:26.269072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:20:39.976731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.266492+0000","last_change":"2024-09-05T02:00:21.416611+0000","last_active":"2024-09-05T02:01:26.266492+0000","last_peered":"2024-09-05T02:01:26.266492+0000","last_clean":"2024-09-05T02:01:26.266492+0000","last_became_active":"2024-09-05T02:00:21.416152+0000","last_became_peered":"2024-09-05T02:00:21.416152+0000","last_unstale":"2024-09-05T02:01:26.266492+0000","last_undegraded":"2024-09-05T02:01:26.266492+0000","last_fullsized":"2024-09-05T02:01:26.266492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:20.670383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507763+0000","last_change":"2024-09-05T02:00:14.359015+0000","last_active":"2024-09-05T02:01:26.507763+0000","last_peered":"2024-09-05T02:01:26.507763+0000","last_clean":"2024-09-05T02:01:26.507763+0000","last_became_active":"2024-09-05T02:00:14.358642+0000","last_became_peered":"2024-09-05T02:00:14.358642+0000","last_unstale":"2024-09-05T02:01:26.507763+0000","last_undegraded":"2024-09-05T02:01:26.507763+0000","last_fullsized":"2024-09-05T02:01:26.507763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:18:03.723033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265712+0000","last_change":"2024-09-05T02:00:14.362391+0000","last_active":"2024-09-05T02:01:26.265712+0000","last_peered":"2024-09-05T02:01:26.265712+0000","last_clean":"2024-09-05T02:01:26.265712+0000","last_became_active":"2024-09-05T02:00:14.362007+0000","last_became_peered":"2024-09-05T02:00:14.362007+0000","last_unstale":"2024-09-05T02:01:26.265712+0000","last_undegraded":"2024-09-05T02:01:26.265712+0000","last_fullsized":"2024-09-05T02:01:26.265712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:39:02.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.269863+0000","last_change":"2024-09-05T02:01:26.269863+0000","last_active":"2024-09-05T02:01:26.269863+0000","last_peered":"2024-09-05T02:01:26.269863+0000","last_clean":"2024-09-05T02:01:26.269863+0000","last_became_active":"2024-09-05T02:01:26.269451+0000","last_became_peered":"2024-09-05T02:01:26.269451+0000","last_unstale":"2024-09-05T02:01:26.269863+0000","last_undegraded":"2024-09-05T02:01:26.269863+0000","last_fullsized":"2024-09-05T02:01:26.269863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:11.900726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267767+0000","last_change":"2024-09-05T02:00:14.357820+0000","last_active":"2024-09-05T02:01:26.267767+0000","last_peered":"2024-09-05T02:01:26.267767+0000","last_clean":"2024-09-05T02:01:26.267767+0000","last_became_active":"2024-09-05T02:00:14.357320+0000","last_became_peered":"2024-09-05T02:00:14.357320+0000","last_unstale":"2024-09-05T02:01:26.267767+0000","last_undegraded":"2024-09-05T02:01:26.267767+0000","last_fullsized":"2024-09-05T02:01:26.267767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:39.105192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281273+0000","last_change":"2024-09-05T02:00:14.356726+0000","last_active":"2024-09-05T02:01:26.281273+0000","last_peered":"2024-09-05T02:01:26.281273+0000","last_clean":"2024-09-05T02:01:26.281273+0000","last_became_active":"2024-09-05T02:00:14.356452+0000","last_became_peered":"2024-09-05T02:00:14.356452+0000","last_unstale":"2024-09-05T02:01:26.281273+0000","last_undegraded":"2024-09-05T02:01:26.281273+0000","last_fullsized":"2024-09-05T02:01:26.281273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:34:36.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507691+0000","last_change":"2024-09-05T02:00:14.357645+0000","last_active":"2024-09-05T02:01:26.507691+0000","last_peered":"2024-09-05T02:01:26.507691+0000","last_clean":"2024-09-05T02:01:26.507691+0000","last_became_active":"2024-09-05T02:00:14.357330+0000","last_became_peered":"2024-09-05T02:00:14.357330+0000","last_unstale":"2024-09-05T02:01:26.507691+0000","last_undegraded":"2024-09-05T02:01:26.507691+0000","last_fullsized":"2024-09-05T02:01:26.507691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:42.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267354+0000","last_change":"2024-09-05T02:00:14.363413+0000","last_active":"2024-09-05T02:01:26.267354+0000","last_peered":"2024-09-05T02:01:26.267354+0000","last_clean":"2024-09-05T02:01:26.267354+0000","last_became_active":"2024-09-05T02:00:14.362821+0000","last_became_peered":"2024-09-05T02:00:14.362821+0000","last_unstale":"2024-09-05T02:01:26.267354+0000","last_undegraded":"2024-09-05T02:01:26.267354+0000","last_fullsized":"2024-09-05T02:01:26.267354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:49:39.084171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507154+0000","last_change":"2024-09-05T02:00:14.366581+0000","last_active":"2024-09-05T02:01:26.507154+0000","last_peered":"2024-09-05T02:01:26.507154+0000","last_clean":"2024-09-05T02:01:26.507154+0000","last_became_active":"2024-09-05T02:00:14.366287+0000","last_became_peered":"2024-09-05T02:00:14.366287+0000","last_unstale":"2024-09-05T02:01:26.507154+0000","last_undegraded":"2024-09-05T02:01:26.507154+0000","last_fullsized":"2024-09-05T02:01:26.507154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:36:57.143489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268446+0000","last_change":"2024-09-05T02:00:14.360311+0000","last_active":"2024-09-05T02:01:26.268446+0000","last_peered":"2024-09-05T02:01:26.268446+0000","last_clean":"2024-09-05T02:01:26.268446+0000","last_became_active":"2024-09-05T02:00:14.360028+0000","last_became_peered":"2024-09-05T02:00:14.360028+0000","last_unstale":"2024-09-05T02:01:26.268446+0000","last_undegraded":"2024-09-05T02:01:26.268446+0000","last_fullsized":"2024-09-05T02:01:26.268446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:35:38.804555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.506952+0000","last_change":"2024-09-05T02:00:14.365853+0000","last_active":"2024-09-05T02:01:26.506952+0000","last_peered":"2024-09-05T02:01:26.506952+0000","last_clean":"2024-09-05T02:01:26.506952+0000","last_became_active":"2024-09-05T02:00:14.365549+0000","last_became_peered":"2024-09-05T02:00:14.365549+0000","last_unstale":"2024-09-05T02:01:26.506952+0000","last_undegraded":"2024-09-05T02:01:26.506952+0000","last_fullsized":"2024-09-05T02:01:26.506952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:58:45.897592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507532+0000","last_change":"2024-09-05T02:00:14.367357+0000","last_active":"2024-09-05T02:01:26.507532+0000","last_peered":"2024-09-05T02:01:26.507532+0000","last_clean":"2024-09-05T02:01:26.507532+0000","last_became_active":"2024-09-05T02:00:14.366794+0000","last_became_peered":"2024-09-05T02:00:14.366794+0000","last_unstale":"2024-09-05T02:01:26.507532+0000","last_undegraded":"2024-09-05T02:01:26.507532+0000","last_fullsized":"2024-09-05T02:01:26.507532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:40:51.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281146+0000","last_change":"2024-09-05T02:00:14.356991+0000","last_active":"2024-09-05T02:01:26.281146+0000","last_peered":"2024-09-05T02:01:26.281146+0000","last_clean":"2024-09-05T02:01:26.281146+0000","last_became_active":"2024-09-05T02:00:14.356662+0000","last_became_peered":"2024-09-05T02:00:14.356662+0000","last_unstale":"2024-09-05T02:01:26.281146+0000","last_undegraded":"2024-09-05T02:01:26.281146+0000","last_fullsized":"2024-09-05T02:01:26.281146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:39:53.087448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507629+0000","last_change":"2024-09-05T02:00:14.357750+0000","last_active":"2024-09-05T02:01:26.507629+0000","last_peered":"2024-09-05T02:01:26.507629+0000","last_clean":"2024-09-05T02:01:26.507629+0000","last_became_active":"2024-09-05T02:00:14.357413+0000","last_became_peered":"2024-09-05T02:00:14.357413+0000","last_unstale":"2024-09-05T02:01:26.507629+0000","last_undegraded":"2024-09-05T02:01:26.507629+0000","last_fullsized":"2024-09-05T02:01:26.507629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:29:47.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.266873+0000","last_change":"2024-09-05T02:00:14.364068+0000","last_active":"2024-09-05T02:01:26.266873+0000","last_peered":"2024-09-05T02:01:26.266873+0000","last_clean":"2024-09-05T02:01:26.266873+0000","last_became_active":"2024-09-05T02:00:14.363674+0000","last_became_peered":"2024-09-05T02:00:14.363674+0000","last_unstale":"2024-09-05T02:01:26.266873+0000","last_undegraded":"2024-09-05T02:01:26.266873+0000","last_fullsized":"2024-09-05T02:01:26.266873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:28:00.788668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265726+0000","last_change":"2024-09-05T02:00:14.355152+0000","last_active":"2024-09-05T02:01:26.265726+0000","last_peered":"2024-09-05T02:01:26.265726+0000","last_clean":"2024-09-05T02:01:26.265726+0000","last_became_active":"2024-09-05T02:00:14.354874+0000","last_became_peered":"2024-09-05T02:00:14.354874+0000","last_unstale":"2024-09-05T02:01:26.265726+0000","last_undegraded":"2024-09-05T02:01:26.265726+0000","last_fullsized":"2024-09-05T02:01:26.265726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:14:17.069049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508040+0000","last_change":"2024-09-05T02:00:14.360241+0000","last_active":"2024-09-05T02:01:26.508040+0000","last_peered":"2024-09-05T02:01:26.508040+0000","last_clean":"2024-09-05T02:01:26.508040+0000","last_became_active":"2024-09-05T02:00:14.359648+0000","last_became_peered":"2024-09-05T02:00:14.359648+0000","last_unstale":"2024-09-05T02:01:26.508040+0000","last_undegraded":"2024-09-05T02:01:26.508040+0000","last_fullsized":"2024-09-05T02:01:26.508040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:15.060575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.507514+0000","last_change":"2024-09-05T02:00:14.366094+0000","last_active":"2024-09-05T02:01:26.507514+0000","last_peered":"2024-09-05T02:01:26.507514+0000","last_clean":"2024-09-05T02:01:26.507514+0000","last_became_active":"2024-09-05T02:00:14.365788+0000","last_became_peered":"2024-09-05T02:00:14.365788+0000","last_unstale":"2024-09-05T02:01:26.507514+0000","last_undegraded":"2024-09-05T02:01:26.507514+0000","last_fullsized":"2024-09-05T02:01:26.507514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:23:00.909487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267635+0000","last_change":"2024-09-05T02:00:14.364651+0000","last_active":"2024-09-05T02:01:26.267635+0000","last_peered":"2024-09-05T02:01:26.267635+0000","last_clean":"2024-09-05T02:01:26.267635+0000","last_became_active":"2024-09-05T02:00:14.364365+0000","last_became_peered":"2024-09-05T02:00:14.364365+0000","last_unstale":"2024-09-05T02:01:26.267635+0000","last_undegraded":"2024-09-05T02:01:26.267635+0000","last_fullsized":"2024-09-05T02:01:26.267635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:14.978382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267589+0000","last_change":"2024-09-05T02:00:14.364761+0000","last_active":"2024-09-05T02:01:26.267589+0000","last_peered":"2024-09-05T02:01:26.267589+0000","last_clean":"2024-09-05T02:01:26.267589+0000","last_became_active":"2024-09-05T02:00:14.364204+0000","last_became_peered":"2024-09-05T02:00:14.364204+0000","last_unstale":"2024-09-05T02:01:26.267589+0000","last_undegraded":"2024-09-05T02:01:26.267589+0000","last_fullsized":"2024-09-05T02:01:26.267589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:16:51.076304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.281131+0000","last_change":"2024-09-05T02:00:14.357632+0000","last_active":"2024-09-05T02:01:26.281131+0000","last_peered":"2024-09-05T02:01:26.281131+0000","last_clean":"2024-09-05T02:01:26.281131+0000","last_became_active":"2024-09-05T02:00:14.357348+0000","last_became_peered":"2024-09-05T02:00:14.357348+0000","last_unstale":"2024-09-05T02:01:26.281131+0000","last_undegraded":"2024-09-05T02:01:26.281131+0000","last_fullsized":"2024-09-05T02:01:26.281131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:11.163292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268194+0000","last_change":"2024-09-05T02:00:14.361484+0000","last_active":"2024-09-05T02:01:26.268194+0000","last_peered":"2024-09-05T02:01:26.268194+0000","last_clean":"2024-09-05T02:01:26.268194+0000","last_became_active":"2024-09-05T02:00:14.361156+0000","last_became_peered":"2024-09-05T02:00:14.361156+0000","last_unstale":"2024-09-05T02:01:26.268194+0000","last_undegraded":"2024-09-05T02:01:26.268194+0000","last_fullsized":"2024-09-05T02:01:26.268194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:40.124733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268635+0000","last_change":"2024-09-05T02:00:14.358325+0000","last_active":"2024-09-05T02:01:26.268635+0000","last_peered":"2024-09-05T02:01:26.268635+0000","last_clean":"2024-09-05T02:01:26.268635+0000","last_became_active":"2024-09-05T02:00:14.357993+0000","last_became_peered":"2024-09-05T02:00:14.357993+0000","last_unstale":"2024-09-05T02:01:26.268635+0000","last_undegraded":"2024-09-05T02:01:26.268635+0000","last_fullsized":"2024-09-05T02:01:26.268635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:18.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.266781+0000","last_change":"2024-09-05T02:00:14.365291+0000","last_active":"2024-09-05T02:01:26.266781+0000","last_peered":"2024-09-05T02:01:26.266781+0000","last_clean":"2024-09-05T02:01:26.266781+0000","last_became_active":"2024-09-05T02:00:14.364937+0000","last_became_peered":"2024-09-05T02:00:14.364937+0000","last_unstale":"2024-09-05T02:01:26.266781+0000","last_undegraded":"2024-09-05T02:01:26.266781+0000","last_fullsized":"2024-09-05T02:01:26.266781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:42:57.712651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508067+0000","last_change":"2024-09-05T02:00:14.367603+0000","last_active":"2024-09-05T02:01:26.508067+0000","last_peered":"2024-09-05T02:01:26.508067+0000","last_clean":"2024-09-05T02:01:26.508067+0000","last_became_active":"2024-09-05T02:00:14.367267+0000","last_became_peered":"2024-09-05T02:00:14.367267+0000","last_unstale":"2024-09-05T02:01:26.508067+0000","last_undegraded":"2024-09-05T02:01:26.508067+0000","last_fullsized":"2024-09-05T02:01:26.508067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:51:34.595898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.506938+0000","last_change":"2024-09-05T02:00:14.369499+0000","last_active":"2024-09-05T02:01:26.506938+0000","last_peered":"2024-09-05T02:01:26.506938+0000","last_clean":"2024-09-05T02:01:26.506938+0000","last_became_active":"2024-09-05T02:00:14.369169+0000","last_became_peered":"2024-09-05T02:00:14.369169+0000","last_unstale":"2024-09-05T02:01:26.506938+0000","last_undegraded":"2024-09-05T02:01:26.506938+0000","last_fullsized":"2024-09-05T02:01:26.506938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:17.161204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267502+0000","last_change":"2024-09-05T02:00:14.360320+0000","last_active":"2024-09-05T02:01:26.267502+0000","last_peered":"2024-09-05T02:01:26.267502+0000","last_clean":"2024-09-05T02:01:26.267502+0000","last_became_active":"2024-09-05T02:00:14.359991+0000","last_became_peered":"2024-09-05T02:00:14.359991+0000","last_unstale":"2024-09-05T02:01:26.267502+0000","last_undegraded":"2024-09-05T02:01:26.267502+0000","last_fullsized":"2024-09-05T02:01:26.267502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:03.721863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.265823+0000","last_change":"2024-09-05T02:00:14.364660+0000","last_active":"2024-09-05T02:01:26.265823+0000","last_peered":"2024-09-05T02:01:26.265823+0000","last_clean":"2024-09-05T02:01:26.265823+0000","last_became_active":"2024-09-05T02:00:14.364313+0000","last_became_peered":"2024-09-05T02:00:14.364313+0000","last_unstale":"2024-09-05T02:01:26.265823+0000","last_undegraded":"2024-09-05T02:01:26.265823+0000","last_fullsized":"2024-09-05T02:01:26.265823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:47.496063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.268892+0000","last_change":"2024-09-05T02:00:14.359853+0000","last_active":"2024-09-05T02:01:26.268892+0000","last_peered":"2024-09-05T02:01:26.268892+0000","last_clean":"2024-09-05T02:01:26.268892+0000","last_became_active":"2024-09-05T02:00:14.359543+0000","last_became_peered":"2024-09-05T02:00:14.359543+0000","last_unstale":"2024-09-05T02:01:26.268892+0000","last_undegraded":"2024-09-05T02:01:26.268892+0000","last_fullsized":"2024-09-05T02:01:26.268892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:30:29.415492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.270888+0000","last_change":"2024-09-05T02:01:26.270888+0000","last_active":"2024-09-05T02:01:26.270888+0000","last_peered":"2024-09-05T02:01:26.270888+0000","last_clean":"2024-09-05T02:01:26.270888+0000","last_became_active":"2024-09-05T02:01:26.270607+0000","last_became_peered":"2024-09-05T02:01:26.270607+0000","last_unstale":"2024-09-05T02:01:26.270888+0000","last_undegraded":"2024-09-05T02:01:26.270888+0000","last_fullsized":"2024-09-05T02:01:26.270888+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:16.563521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267574+0000","last_change":"2024-09-05T02:00:14.367191+0000","last_active":"2024-09-05T02:01:26.267574+0000","last_peered":"2024-09-05T02:01:26.267574+0000","last_clean":"2024-09-05T02:01:26.267574+0000","last_became_active":"2024-09-05T02:00:14.366831+0000","last_became_peered":"2024-09-05T02:00:14.366831+0000","last_unstale":"2024-09-05T02:01:26.267574+0000","last_undegraded":"2024-09-05T02:01:26.267574+0000","last_fullsized":"2024-09-05T02:01:26.267574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:55.690642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.267770+0000","last_change":"2024-09-05T02:00:14.355774+0000","last_active":"2024-09-05T02:01:26.267770+0000","last_peered":"2024-09-05T02:01:26.267770+0000","last_clean":"2024-09-05T02:01:26.267770+0000","last_became_active":"2024-09-05T02:00:14.355293+0000","last_became_peered":"2024-09-05T02:00:14.355293+0000","last_unstale":"2024-09-05T02:01:26.267770+0000","last_undegraded":"2024-09-05T02:01:26.267770+0000","last_fullsized":"2024-09-05T02:01:26.267770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:07:00.547820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T02:01:26.508355+0000","last_change":"2024-09-05T02:00:14.359978+0000","last_active":"2024-09-05T02:01:26.508355+0000","last_peered":"2024-09-05T02:01:26.508355+0000","last_clean":"2024-09-05T02:01:26.508355+0000","last_became_active":"2024-09-05T02:00:14.359209+0000","last_became_peered":"2024-09-05T02:00:14.359209+0000","last_unstale":"2024-09-05T02:01:26.508355+0000","last_undegraded":"2024-09-05T02:01:26.508355+0000","last_fullsized":"2024-09-05T02:01:26.508355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:27.848906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69284,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":545,"num_read_kb":346,"num_write":249,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579937,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1007616,"data_stored":497661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808870,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1699840,"data_stored":1146160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":501294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201329,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":497576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430261,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1626112,"data_stored":1094075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":492702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36948,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708204,"statfs":{"total":95995035648,"available":95957200896,"internally_reserved":0,"allocated":1634304,"data_stored":1089657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1024000,"data_stored":496888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1019904,"data_stored":492410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1085440,"data_stored":556702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1073152,"data_stored":555808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":494551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T02:01:54.614 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T02:01:54.614 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-05T02:01:54.614 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-05T02:01:54.615 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph health --format=json 2024-09-05T02:01:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:54 smithi167 ceph-mon[35122]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:01:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:54 smithi139 ceph-mon[34262]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:01:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:55.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 -- 172.21.15.103:0/728326739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 msgr2=0x7fc518107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:55.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 --2- 172.21.15.103:0/728326739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 0x7fc518107bc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc500009930 tx=0x7fc50002a420 comp rx=0 tx=0).stop 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 -- 172.21.15.103:0/728326739 shutdown_connections 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 --2- 172.21.15.103:0/728326739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc518108300 0x7fc51810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 --2- 172.21.15.103:0/728326739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 0x7fc518107bc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 --2- 172.21.15.103:0/728326739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc518102dd0 0x7fc5181031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 -- 172.21.15.103:0/728326739 >> 172.21.15.103:0/728326739 conn(0x7fc5180fc7d0 msgr2=0x7fc5180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 -- 172.21.15.103:0/728326739 shutdown_connections 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 -- 172.21.15.103:0/728326739 wait complete. 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.401+0000 7fc51d46c640 1 Processor -- start 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc51d46c640 1 -- start start 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc51d46c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc518102dd0 0x7fc518198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc51d46c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 0x7fc518198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc51d46c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc518108300 0x7fc51819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc51d46c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc518111f80 con 0x7fc518103780 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc51d46c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc518111e00 con 0x7fc518102dd0 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc51d46c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc518112100 con 0x7fc518108300 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc5167fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 0x7fc518198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc5167fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 0x7fc518198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60762/0 (socket says 172.21.15.103:60762) 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc5167fc640 1 -- 172.21.15.103:0/3809941425 learned_addr learned my addr 172.21.15.103:0/3809941425 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc516ffd640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc518102dd0 0x7fc518198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc5167fc640 1 -- 172.21.15.103:0/3809941425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc518108300 msgr2=0x7fc51819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc5167fc640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc518108300 0x7fc51819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc5167fc640 1 -- 172.21.15.103:0/3809941425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc518102dd0 msgr2=0x7fc518198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc5167fc640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc518102dd0 0x7fc518198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:55.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc5167fc640 1 -- 172.21.15.103:0/3809941425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51819d530 con 0x7fc518103780 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.402+0000 7fc516ffd640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc518102dd0 0x7fc518198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.403+0000 7fc5167fc640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 0x7fc518198ac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc500009230 tx=0x7fc500004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.403+0000 7fc50ffff640 1 -- 172.21.15.103:0/3809941425 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50003f030 con 0x7fc518103780 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.403+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51819d7c0 con 0x7fc518103780 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.403+0000 7fc50ffff640 1 -- 172.21.15.103:0/3809941425 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5000035f0 con 0x7fc518103780 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.403+0000 7fc50ffff640 1 -- 172.21.15.103:0/3809941425 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500035070 con 0x7fc518103780 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.403+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5181a5100 con 0x7fc518103780 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.404+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5181031b0 con 0x7fc518103780 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.405+0000 7fc50ffff640 1 -- 172.21.15.103:0/3809941425 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc500003aa0 con 0x7fc518103780 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.405+0000 7fc50ffff640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc5040a2c10 0x7fc5040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.406+0000 7fc516ffd640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc5040a2c10 0x7fc5040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:55.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.406+0000 7fc50ffff640 1 -- 172.21.15.103:0/3809941425 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fc5000eff80 con 0x7fc518103780 2024-09-05T02:01:55.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.406+0000 7fc516ffd640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc5040a2c10 0x7fc5040a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc510009930 tx=0x7fc510008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:55.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.410+0000 7fc50ffff640 1 -- 172.21.15.103:0/3809941425 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5000bca20 con 0x7fc518103780 2024-09-05T02:01:55.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.611+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc51810a290 con 0x7fc518103780 2024-09-05T02:01:55.613 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:55 smithi103 ceph-mon[30902]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:55.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.612+0000 7fc50ffff640 1 -- 172.21.15.103:0/3809941425 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc5000bc320 con 0x7fc518103780 2024-09-05T02:01:55.614 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:55.614 INFO:teuthology.orchestra.run.smithi103.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-05T02:01:55.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc5040a2c10 msgr2=0x7fc5040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:55.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc5040a2c10 0x7fc5040a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc510009930 tx=0x7fc510008040 comp rx=0 tx=0).stop 2024-09-05T02:01:55.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 msgr2=0x7fc518198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:55.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 0x7fc518198ac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc500009230 tx=0x7fc500004270 comp rx=0 tx=0).stop 2024-09-05T02:01:55.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 shutdown_connections 2024-09-05T02:01:55.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc5040a2c10 0x7fc5040a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:55.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc518108300 0x7fc51819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:55.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc518103780 0x7fc518198ac0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:55.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 --2- 172.21.15.103:0/3809941425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc518102dd0 0x7fc518198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:55.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 >> 172.21.15.103:0/3809941425 conn(0x7fc5180fc7d0 msgr2=0x7fc518105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:55.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 shutdown_connections 2024-09-05T02:01:55.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:55.614+0000 7fc51d46c640 1 -- 172.21.15.103:0/3809941425 wait complete. 2024-09-05T02:01:55.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:55 smithi167 ceph-mon[35122]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:55.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:55 smithi139 ceph-mon[34262]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:01:56.432 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-05T02:01:56.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:01:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:56 smithi167 ceph-mon[35122]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:01:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:56 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3809941425' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T02:01:56.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:56 smithi139 ceph-mon[34262]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:01:56.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:56 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3809941425' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T02:01:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:56 smithi103 ceph-mon[30902]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:01:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:56 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3809941425' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T02:01:56.680 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.186+0000 7f0efc538640 1 -- 172.21.15.103:0/1178501843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4106f10 msgr2=0x7f0ef410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.186+0000 7f0efc538640 1 --2- 172.21.15.103:0/1178501843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4106f10 0x7f0ef410ea90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009ca0 tx=0x7f0eec02a8c0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 -- 172.21.15.103:0/1178501843 shutdown_connections 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 --2- 172.21.15.103:0/1178501843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4106f10 0x7f0ef410ea90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 --2- 172.21.15.103:0/1178501843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef4102790 0x7f0ef41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 --2- 172.21.15.103:0/1178501843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ef4101de0 0x7f0ef41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 -- 172.21.15.103:0/1178501843 >> 172.21.15.103:0/1178501843 conn(0x7f0ef40fd8c0 msgr2=0x7f0ef40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 -- 172.21.15.103:0/1178501843 shutdown_connections 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 -- 172.21.15.103:0/1178501843 wait complete. 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 Processor -- start 2024-09-05T02:01:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 -- start start 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4101de0 0x7f0ef41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ef4102790 0x7f0ef4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef4106f10 0x7f0ef419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4111e60 con 0x7f0ef4101de0 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4111ce0 con 0x7f0ef4106f10 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.187+0000 7f0efc538640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4111fe0 con 0x7f0ef4102790 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efa2d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4101de0 0x7f0ef41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efa2d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4101de0 0x7f0ef41983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60770/0 (socket says 172.21.15.103:60770) 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efa2d4640 1 -- 172.21.15.103:0/2071433254 learned_addr learned my addr 172.21.15.103:0/2071433254 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efaad5640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef4106f10 0x7f0ef419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0ef9ad3640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ef4102790 0x7f0ef4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efa2d4640 1 -- 172.21.15.103:0/2071433254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ef4102790 msgr2=0x7f0ef4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:57.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efa2d4640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ef4102790 0x7f0ef4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efa2d4640 1 -- 172.21.15.103:0/2071433254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef4106f10 msgr2=0x7f0ef419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efa2d4640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef4106f10 0x7f0ef419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efa2d4640 1 -- 172.21.15.103:0/2071433254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef419d370 con 0x7f0ef4101de0 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efaad5640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef4106f10 0x7f0ef419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efa2d4640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4101de0 0x7f0ef41983c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c970 tx=0x7f0ef000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0eeb7fe640 1 -- 172.21.15.103:0/2071433254 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0007bb0 con 0x7f0ef4101de0 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0eeb7fe640 1 -- 172.21.15.103:0/2071433254 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ef0007d50 con 0x7f0ef4101de0 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0eeb7fe640 1 -- 172.21.15.103:0/2071433254 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000f450 con 0x7f0ef4101de0 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef419d660 con 0x7f0ef4101de0 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.188+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef41a4f40 con 0x7f0ef4101de0 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.189+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef41021c0 con 0x7f0ef4101de0 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.190+0000 7f0eeb7fe640 1 -- 172.21.15.103:0/2071433254 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0ef0004f00 con 0x7f0ef4101de0 2024-09-05T02:01:57.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.191+0000 7f0eeb7fe640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0ee00a2c10 0x7f0ee00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:57.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.191+0000 7f0eeb7fe640 1 -- 172.21.15.103:0/2071433254 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0ef00cf6e0 con 0x7f0ef4101de0 2024-09-05T02:01:57.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.191+0000 7f0ef9ad3640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0ee00a2c10 0x7f0ee00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:57.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.192+0000 7f0ef9ad3640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0ee00a2c10 0x7f0ee00a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40fdce0 tx=0x7f0ee4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:57.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.194+0000 7f0eeb7fe640 1 -- 172.21.15.103:0/2071433254 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ef0014030 con 0x7f0ef4101de0 2024-09-05T02:01:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.395+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ef4199720 con 0x7f0ef4101de0 2024-09-05T02:01:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.396+0000 7f0eeb7fe640 1 -- 172.21.15.103:0/2071433254 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7f0ef4199720 con 0x7f0ef4101de0 2024-09-05T02:01:57.398 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:57.398 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":28,"btime":"2024-09-05T02:00:59:232233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:00:59.232218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:01:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 28 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0ee00a2c10 msgr2=0x7f0ee00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0ee00a2c10 0x7f0ee00a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40fdce0 tx=0x7f0ee4008040 comp rx=0 tx=0).stop 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4101de0 msgr2=0x7f0ef41983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4101de0 0x7f0ef41983c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c970 tx=0x7f0ef000ce30 comp rx=0 tx=0).stop 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 shutdown_connections 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef4106f10 0x7f0ef419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0ee00a2c10 0x7f0ee00a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ef4102790 0x7f0ef4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 --2- 172.21.15.103:0/2071433254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef4101de0 0x7f0ef41983c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 >> 172.21.15.103:0/2071433254 conn(0x7f0ef40fd8c0 msgr2=0x7f0ef40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 shutdown_connections 2024-09-05T02:01:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:57.398+0000 7f0efc538640 1 -- 172.21.15.103:0/2071433254 wait complete. 2024-09-05T02:01:58.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:01:58.511 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:01:58.632 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:58 smithi103 ceph-mon[30902]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:01:58.632 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:58 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2071433254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:01:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:58 smithi167 ceph-mon[35122]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:01:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:58 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2071433254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:01:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:58 smithi139 ceph-mon[34262]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:01:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:58 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2071433254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:01:59.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 -- 172.21.15.103:0/2145717794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 msgr2=0x7f2268100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:59.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 --2- 172.21.15.103:0/2145717794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 0x7f2268100ee0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2250009930 tx=0x7f225002a420 comp rx=0 tx=0).stop 2024-09-05T02:01:59.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 -- 172.21.15.103:0/2145717794 shutdown_connections 2024-09-05T02:01:59.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 --2- 172.21.15.103:0/2145717794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2268101420 0x7f226810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 --2- 172.21.15.103:0/2145717794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 0x7f2268100ee0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 --2- 172.21.15.103:0/2145717794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2268106a80 0x7f2268106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 -- 172.21.15.103:0/2145717794 >> 172.21.15.103:0/2145717794 conn(0x7f22680fc7a0 msgr2=0x7f22680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:59.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 -- 172.21.15.103:0/2145717794 shutdown_connections 2024-09-05T02:01:59.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 -- 172.21.15.103:0/2145717794 wait complete. 2024-09-05T02:01:59.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.016+0000 7f226dadb640 1 Processor -- start 2024-09-05T02:01:59.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f226dadb640 1 -- start start 2024-09-05T02:01:59.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f226dadb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 0x7f22681982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f226dadb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2268101420 0x7f22681987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f226dadb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2268106a80 0x7f226819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f226dadb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2268111e50 con 0x7f2268100a80 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f226dadb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2268111cd0 con 0x7f2268101420 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f226dadb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2268111fd0 con 0x7f2268106a80 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 0x7f22681982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 0x7f22681982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60796/0 (socket says 172.21.15.103:60796) 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22677fe640 1 -- 172.21.15.103:0/2250222789 learned_addr learned my addr 172.21.15.103:0/2250222789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f2267fff640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2268106a80 0x7f226819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f225ffff640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2268101420 0x7f22681987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22677fe640 1 -- 172.21.15.103:0/2250222789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2268106a80 msgr2=0x7f226819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22677fe640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2268106a80 0x7f226819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22677fe640 1 -- 172.21.15.103:0/2250222789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2268101420 msgr2=0x7f22681987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22677fe640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2268101420 0x7f22681987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22677fe640 1 -- 172.21.15.103:0/2250222789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226819d260 con 0x7f2268100a80 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f225ffff640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2268101420 0x7f22681987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22677fe640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 0x7f22681982b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f225800c920 tx=0x7f225800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.017+0000 7f22657fa640 1 -- 172.21.15.103:0/2250222789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258007c70 con 0x7f2268100a80 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.018+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226819d550 con 0x7f2268100a80 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.018+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22681a4e30 con 0x7f2268100a80 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.018+0000 7f22657fa640 1 -- 172.21.15.103:0/2250222789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2258007e10 con 0x7f2268100a80 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.018+0000 7f22657fa640 1 -- 172.21.15.103:0/2250222789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258018820 con 0x7f2268100a80 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.019+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2268100ee0 con 0x7f2268100a80 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.020+0000 7f22657fa640 1 -- 172.21.15.103:0/2250222789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f22580189c0 con 0x7f2268100a80 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.020+0000 7f22657fa640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f22540a2c90 0x7f22540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:01:59.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.020+0000 7f225ffff640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f22540a2c90 0x7f22540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:01:59.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.020+0000 7f22657fa640 1 -- 172.21.15.103:0/2250222789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f22580cf660 con 0x7f2268100a80 2024-09-05T02:01:59.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.021+0000 7f225ffff640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f22540a2c90 0x7f22540a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2250000c00 tx=0x7f2250031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:01:59.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.024+0000 7f22657fa640 1 -- 172.21.15.103:0/2250222789 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2258014030 con 0x7f2268100a80 2024-09-05T02:01:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.224+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22681a5100 con 0x7f2268100a80 2024-09-05T02:01:59.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.225+0000 7f22657fa640 1 -- 172.21.15.103:0/2250222789 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7f225800de20 con 0x7f2268100a80 2024-09-05T02:01:59.227 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:01:59.227 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":28,"btime":"2024-09-05T02:00:59:232233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:00:59.232218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:01:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 28 2024-09-05T02:01:59.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.227+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f22540a2c90 msgr2=0x7f22540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:59.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f22540a2c90 0x7f22540a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2250000c00 tx=0x7f2250031400 comp rx=0 tx=0).stop 2024-09-05T02:01:59.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 msgr2=0x7f22681982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:01:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 0x7f22681982b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f225800c920 tx=0x7f225800cde0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 shutdown_connections 2024-09-05T02:01:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f22540a2c90 0x7f22540a5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2268106a80 0x7f226819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2268101420 0x7f22681987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 --2- 172.21.15.103:0/2250222789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2268100a80 0x7f22681982b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:01:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 >> 172.21.15.103:0/2250222789 conn(0x7f22680fc7a0 msgr2=0x7f22680fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:01:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 shutdown_connections 2024-09-05T02:01:59.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:01:59.228+0000 7f226dadb640 1 -- 172.21.15.103:0/2250222789 wait complete. 2024-09-05T02:01:59.557 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:01:59 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2250222789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:01:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:01:59 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2250222789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:01:59.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:01:59 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2250222789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:01:59.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:02:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:00.523 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:00 smithi103 ceph-mon[30902]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:02:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:00 smithi167 ceph-mon[35122]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:02:00.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:00 smithi139 ceph-mon[34262]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:02:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.774+0000 7f600ea33640 1 -- 172.21.15.103:0/699727706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008101de0 msgr2=0x7f60081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.774+0000 7f600ea33640 1 --2- 172.21.15.103:0/699727706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008101de0 0x7f60081021c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009900 tx=0x7f5ff802a410 comp rx=0 tx=0).stop 2024-09-05T02:02:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.775+0000 7f600ea33640 1 -- 172.21.15.103:0/699727706 shutdown_connections 2024-09-05T02:02:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.775+0000 7f600ea33640 1 --2- 172.21.15.103:0/699727706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6008106f10 0x7f600810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.775+0000 7f600ea33640 1 --2- 172.21.15.103:0/699727706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6008102790 0x7f60081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.775+0000 7f600ea33640 1 --2- 172.21.15.103:0/699727706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008101de0 0x7f60081021c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.775+0000 7f600ea33640 1 -- 172.21.15.103:0/699727706 >> 172.21.15.103:0/699727706 conn(0x7f60080fd8c0 msgr2=0x7f60080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.775+0000 7f600ea33640 1 -- 172.21.15.103:0/699727706 shutdown_connections 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.775+0000 7f600ea33640 1 -- 172.21.15.103:0/699727706 wait complete. 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.775+0000 7f600ea33640 1 Processor -- start 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600ea33640 1 -- start start 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600ea33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6008101de0 0x7f60080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600ea33640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6008102790 0x7f6008074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600ea33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008106f10 0x7f6008074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600ea33640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6008111ec0 con 0x7f6008106f10 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600ea33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6008111d40 con 0x7f6008101de0 2024-09-05T02:02:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600ea33640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6008112040 con 0x7f6008102790 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600cfd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008106f10 0x7f6008074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600cfd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008106f10 0x7f6008074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34794/0 (socket says 172.21.15.103:34794) 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600cfd0640 1 -- 172.21.15.103:0/921835154 learned_addr learned my addr 172.21.15.103:0/921835154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f5ffffff640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6008102790 0x7f6008074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f6007fff640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6008101de0 0x7f60080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600cfd0640 1 -- 172.21.15.103:0/921835154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6008102790 msgr2=0x7f6008074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600cfd0640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6008102790 0x7f6008074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600cfd0640 1 -- 172.21.15.103:0/921835154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6008101de0 msgr2=0x7f60080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600cfd0640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6008101de0 0x7f60080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600cfd0640 1 -- 172.21.15.103:0/921835154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60081a69e0 con 0x7f6008106f10 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f6007fff640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6008101de0 0x7f60080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.776+0000 7f600cfd0640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008106f10 0x7f6008074f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f600000b970 tx=0x7f600000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.777+0000 7f6005ffb640 1 -- 172.21.15.103:0/921835154 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000019070 con 0x7f6008106f10 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.777+0000 7f6005ffb640 1 -- 172.21.15.103:0/921835154 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6000014440 con 0x7f6008106f10 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.777+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60081a6cd0 con 0x7f6008106f10 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.777+0000 7f6005ffb640 1 -- 172.21.15.103:0/921835154 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000075d0 con 0x7f6008106f10 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.777+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60081a7210 con 0x7f6008106f10 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.778+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6008102240 con 0x7f6008106f10 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.779+0000 7f6005ffb640 1 -- 172.21.15.103:0/921835154 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f600001c070 con 0x7f6008106f10 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.779+0000 7f6005ffb640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5ff40a2c90 0x7f5ff40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.780+0000 7f6007fff640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5ff40a2c90 0x7f5ff40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.780+0000 7f6005ffb640 1 -- 172.21.15.103:0/921835154 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f60000cee00 con 0x7f6008106f10 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.780+0000 7f6007fff640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5ff40a2c90 0x7f5ff40a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8000c00 tx=0x7f5ff8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.783+0000 7f6005ffb640 1 -- 172.21.15.103:0/921835154 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f600009b9b0 con 0x7f6008106f10 2024-09-05T02:02:00.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.951+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f600810a120 con 0x7f6008106f10 2024-09-05T02:02:00.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.952+0000 7f6005ffb640 1 -- 172.21.15.103:0/921835154 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24730 (secure 0 0 0) 0x7f6000095e20 con 0x7f6008106f10 2024-09-05T02:02:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:00.954 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":88,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:01:26.255904+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:02:00.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5ff40a2c90 msgr2=0x7f5ff40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:00.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5ff40a2c90 0x7f5ff40a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8000c00 tx=0x7f5ff8031400 comp rx=0 tx=0).stop 2024-09-05T02:02:00.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008106f10 msgr2=0x7f6008074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:00.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008106f10 0x7f6008074f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f600000b970 tx=0x7f600000bd70 comp rx=0 tx=0).stop 2024-09-05T02:02:00.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 shutdown_connections 2024-09-05T02:02:00.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6008106f10 0x7f6008074f70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:00.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5ff40a2c90 0x7f5ff40a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:00.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6008102790 0x7f6008074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:00.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 --2- 172.21.15.103:0/921835154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6008101de0 0x7f60080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:00.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 >> 172.21.15.103:0/921835154 conn(0x7f60080fd8c0 msgr2=0x7f60080fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:00.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 shutdown_connections 2024-09-05T02:02:00.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:00.954+0000 7f600ea33640 1 -- 172.21.15.103:0/921835154 wait complete. 2024-09-05T02:02:01.404 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:01 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/921835154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:02:01.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:01 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/921835154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:02:01.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:01 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/921835154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:02:01.765 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:02:02.010 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:02.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:02 smithi103 ceph-mon[30902]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:02:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.536+0000 7f5a0aa92640 1 -- 172.21.15.103:0/1376310130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04103660 msgr2=0x7f5a0410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.536+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/1376310130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04103660 0x7f5a0410bb50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009900 tx=0x7f59f002a480 comp rx=0 tx=0).stop 2024-09-05T02:02:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.536+0000 7f5a0aa92640 1 -- 172.21.15.103:0/1376310130 shutdown_connections 2024-09-05T02:02:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.536+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/1376310130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a0410c170 0x7f5a0410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.536+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/1376310130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04103660 0x7f5a0410bb50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.536+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/1376310130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a04102d40 0x7f5a04103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.536+0000 7f5a0aa92640 1 -- 172.21.15.103:0/1376310130 >> 172.21.15.103:0/1376310130 conn(0x7f5a040fc7a0 msgr2=0x7f5a040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.536+0000 7f5a0aa92640 1 -- 172.21.15.103:0/1376310130 shutdown_connections 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.536+0000 7f5a0aa92640 1 -- 172.21.15.103:0/1376310130 wait complete. 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0aa92640 1 Processor -- start 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0aa92640 1 -- start start 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0aa92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04102d40 0x7f5a04198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0aa92640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a04103660 0x7f5a041988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0aa92640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0410c170 0x7f5a0419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0aa92640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a04111e20 con 0x7f5a04102d40 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0aa92640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a04111ca0 con 0x7f5a0410c170 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0aa92640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a04111fa0 con 0x7f5a04103660 2024-09-05T02:02:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0882e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04102d40 0x7f5a04198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0882e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04102d40 0x7f5a04198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34818/0 (socket says 172.21.15.103:34818) 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0882e640 1 -- 172.21.15.103:0/898700791 learned_addr learned my addr 172.21.15.103:0/898700791 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a03fff640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a04103660 0x7f5a041988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0902f640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0410c170 0x7f5a0419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0882e640 1 -- 172.21.15.103:0/898700791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a04103660 msgr2=0x7f5a041988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0882e640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a04103660 0x7f5a041988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0882e640 1 -- 172.21.15.103:0/898700791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0410c170 msgr2=0x7f5a0419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0882e640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0410c170 0x7f5a0419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0882e640 1 -- 172.21.15.103:0/898700791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0419d320 con 0x7f5a04102d40 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.537+0000 7f5a0902f640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0410c170 0x7f5a0419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.538+0000 7f5a0882e640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04102d40 0x7f5a04198370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59f800d950 tx=0x7f59f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.538+0000 7f5a01ffb640 1 -- 172.21.15.103:0/898700791 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8017070 con 0x7f5a04102d40 2024-09-05T02:02:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.538+0000 7f5a01ffb640 1 -- 172.21.15.103:0/898700791 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59f8009c30 con 0x7f5a04102d40 2024-09-05T02:02:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.538+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0419d610 con 0x7f5a04102d40 2024-09-05T02:02:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.538+0000 7f5a01ffb640 1 -- 172.21.15.103:0/898700791 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f800f620 con 0x7f5a04102d40 2024-09-05T02:02:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.538+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a041a4ef0 con 0x7f5a04102d40 2024-09-05T02:02:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.539+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a04103120 con 0x7f5a04102d40 2024-09-05T02:02:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.540+0000 7f5a01ffb640 1 -- 172.21.15.103:0/898700791 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f59f80109a0 con 0x7f5a04102d40 2024-09-05T02:02:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.540+0000 7f5a01ffb640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f59f40a2c90 0x7f59f40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.541+0000 7f5a03fff640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f59f40a2c90 0x7f59f40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.541+0000 7f5a01ffb640 1 -- 172.21.15.103:0/898700791 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f59f80cf0f0 con 0x7f5a04102d40 2024-09-05T02:02:02.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.541+0000 7f5a03fff640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f59f40a2c90 0x7f59f40a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f59f0031440 tx=0x7f59f0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:02.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.544+0000 7f5a01ffb640 1 -- 172.21.15.103:0/898700791 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59f8009de0 con 0x7f5a04102d40 2024-09-05T02:02:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:02 smithi167 ceph-mon[35122]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:02:02.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:02 smithi139 ceph-mon[34262]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:02:02.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.744+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a041071e0 con 0x7f5a04102d40 2024-09-05T02:02:02.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.745+0000 7f5a01ffb640 1 -- 172.21.15.103:0/898700791 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7f59f809bb90 con 0x7f5a04102d40 2024-09-05T02:02:02.747 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:02.747 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":28,"btime":"2024-09-05T02:00:59:232233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:00:59.232218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:02:02.747 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 28 2024-09-05T02:02:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f59f40a2c90 msgr2=0x7f59f40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f59f40a2c90 0x7f59f40a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f59f0031440 tx=0x7f59f0005b50 comp rx=0 tx=0).stop 2024-09-05T02:02:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04102d40 msgr2=0x7f5a04198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04102d40 0x7f5a04198370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59f800d950 tx=0x7f59f800de10 comp rx=0 tx=0).stop 2024-09-05T02:02:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 shutdown_connections 2024-09-05T02:02:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0410c170 0x7f5a0419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f59f40a2c90 0x7f59f40a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a04103660 0x7f5a041988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 --2- 172.21.15.103:0/898700791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a04102d40 0x7f5a04198370 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 >> 172.21.15.103:0/898700791 conn(0x7f5a040fc7a0 msgr2=0x7f5a040fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 shutdown_connections 2024-09-05T02:02:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:02.747+0000 7f5a0aa92640 1 -- 172.21.15.103:0/898700791 wait complete. 2024-09-05T02:02:03.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:03 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/898700791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:03.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T02:00:15.086875+0000', 'modified': '2024-09-05T02:00:59.232218+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24497, 'mds_1': 24466, 'mds_2': 24464, 'mds_3': 24457, 'mds_4': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/1603518036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 1603518036}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 1603518036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/426074448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 426074448}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 426074448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6833/4042514104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 4042514104}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 4042514104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/2095779466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 2095779466}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 2095779466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/1110459707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 1110459707}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 1110459707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.139:6835/3204357322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 3204357322}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 3204357322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.167:6837/3468047781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 3468047781}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3468047781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.139:6837/2336570561', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 2336570561}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 2336570561}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.167:6839/1858062899', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 1858062899}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 1858062899}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.139:6839/3082363979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 3082363979}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 3082363979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24497, 24466, 24464, 24457, 24449]} 2024-09-05T02:02:03.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-05T02:02:03.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.e status 2024-09-05T02:02:03.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:03 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/898700791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:03.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:03 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/898700791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:03.819 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:04.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.331+0000 7fd107b0a640 1 -- 172.21.15.103:0/1748972765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100101df0 msgr2=0x7fd1001021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:04.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.331+0000 7fd107b0a640 1 --2- 172.21.15.103:0/1748972765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100101df0 0x7fd1001021d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009900 tx=0x7fd0fc02a3f0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.331+0000 7fd107b0a640 1 -- 172.21.15.103:0/1748972765 shutdown_connections 2024-09-05T02:02:04.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.331+0000 7fd107b0a640 1 --2- 172.21.15.103:0/1748972765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd100106f20 0x7fd10010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.331+0000 7fd107b0a640 1 --2- 172.21.15.103:0/1748972765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1001027a0 0x7fd1001067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.331+0000 7fd107b0a640 1 --2- 172.21.15.103:0/1748972765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100101df0 0x7fd1001021d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.331+0000 7fd107b0a640 1 -- 172.21.15.103:0/1748972765 >> 172.21.15.103:0/1748972765 conn(0x7fd1000fd8b0 msgr2=0x7fd1000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.331+0000 7fd107b0a640 1 -- 172.21.15.103:0/1748972765 shutdown_connections 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.331+0000 7fd107b0a640 1 -- 172.21.15.103:0/1748972765 wait complete. 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd107b0a640 1 Processor -- start 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd107b0a640 1 -- start start 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd107b0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd100101df0 0x7fd1001983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd107b0a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1001027a0 0x7fd100198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd107b0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100106f20 0x7fd10019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd107b0a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd100111df0 con 0x7fd100106f20 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd107b0a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd100111c70 con 0x7fd100101df0 2024-09-05T02:02:04.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd107b0a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd100111f70 con 0x7fd1001027a0 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1060a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100106f20 0x7fd10019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1060a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100106f20 0x7fd10019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34832/0 (socket says 172.21.15.103:34832) 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1060a7640 1 -- 172.21.15.103:0/63755839 learned_addr learned my addr 172.21.15.103:0/63755839 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1050a5640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1001027a0 0x7fd100198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1058a6640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd100101df0 0x7fd1001983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1060a7640 1 -- 172.21.15.103:0/63755839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1001027a0 msgr2=0x7fd100198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1060a7640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1001027a0 0x7fd100198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1060a7640 1 -- 172.21.15.103:0/63755839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd100101df0 msgr2=0x7fd1001983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1060a7640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd100101df0 0x7fd1001983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.332+0000 7fd1060a7640 1 -- 172.21.15.103:0/63755839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10019d380 con 0x7fd100106f20 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.333+0000 7fd1060a7640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100106f20 0x7fd10019cca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800eaa0 tx=0x7fd0f800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.333+0000 7fd0f6ffd640 1 -- 172.21.15.103:0/63755839 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8004d20 con 0x7fd100106f20 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.333+0000 7fd107b0a640 1 -- 172.21.15.103:0/63755839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10019d670 con 0x7fd100106f20 2024-09-05T02:02:04.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.333+0000 7fd0f6ffd640 1 -- 172.21.15.103:0/63755839 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0f8005020 con 0x7fd100106f20 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.333+0000 7fd0f6ffd640 1 -- 172.21.15.103:0/63755839 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80116b0 con 0x7fd100106f20 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.333+0000 7fd107b0a640 1 -- 172.21.15.103:0/63755839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1001a4f50 con 0x7fd100106f20 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.335+0000 7fd0f6ffd640 1 -- 172.21.15.103:0/63755839 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd0f80051d0 con 0x7fd100106f20 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.335+0000 7fd109ee3740 1 Processor -- start 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd109ee3740 1 -- start start 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082d9d0 0x558e0082ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082e2f0 0x558e00838670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e00838bb0 0x558e0083afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd109ee3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e00794090 con 0x558e0082e2f0 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd109ee3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e0073ba70 con 0x558e00838bb0 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd109ee3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558e00790050 con 0x558e0082d9d0 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1058a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082e2f0 0x558e00838670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd0f6ffd640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1058a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082e2f0 0x558e00838670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34844/0 (socket says 172.21.15.103:34844) 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1058a6640 1 -- 172.21.15.103:0/2189383805 learned_addr learned my addr 172.21.15.103:0/2189383805 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1050a5640 1 --2- 172.21.15.103:0/2189383805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e00838bb0 0x558e0083afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1050a5640 1 --2- 172.21.15.103:0/2189383805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082d9d0 0x558e0082ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1060a7640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1058a6640 1 -- 172.21.15.103:0/2189383805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082d9d0 msgr2=0x558e0082ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1058a6640 1 --2- 172.21.15.103:0/2189383805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082d9d0 0x558e0082ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1058a6640 1 -- 172.21.15.103:0/2189383805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e00838bb0 msgr2=0x558e0083afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1058a6640 1 --2- 172.21.15.103:0/2189383805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e00838bb0 0x558e0083afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd0f6ffd640 1 -- 172.21.15.103:0/63755839 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fd0f80cf730 con 0x7fd100106f20 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.336+0000 7fd1058a6640 1 -- 172.21.15.103:0/2189383805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e00748130 con 0x558e0082e2f0 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.337+0000 7fd1058a6640 1 --2- 172.21.15.103:0/2189383805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082e2f0 0x558e00838670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc02a910 tx=0x7fd0fc032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c2186d549e2dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.337+0000 7fd1060a7640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd0f809ba00 tx=0x7fd0f806a9f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:04.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.337+0000 7fd0ebfff640 1 -- 172.21.15.103:0/2189383805 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc03f070 con 0x558e0082e2f0 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.337+0000 7fd0ebfff640 1 -- 172.21.15.103:0/2189383805 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0fc0316a0 con 0x558e0082e2f0 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.337+0000 7fd0ebfff640 1 -- 172.21.15.103:0/2189383805 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc035070 con 0x558e0082e2f0 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 -- 172.21.15.103:0/2189383805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082e2f0 msgr2=0x558e00838670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 --2- 172.21.15.103:0/2189383805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082e2f0 0x558e00838670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc02a910 tx=0x7fd0fc032cc0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 -- 172.21.15.103:0/2189383805 shutdown_connections 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 --2- 172.21.15.103:0/2189383805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e00838bb0 0x558e0083afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 --2- 172.21.15.103:0/2189383805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082e2f0 0x558e00838670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 --2- 172.21.15.103:0/2189383805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082d9d0 0x558e0082ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 -- 172.21.15.103:0/2189383805 >> 172.21.15.103:0/2189383805 conn(0x558e00829f60 msgr2=0x558e0082a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 -- 172.21.15.103:0/2189383805 shutdown_connections 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 -- 172.21.15.103:0/2189383805 wait complete. 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 -- start start 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 Processor -- start 2024-09-05T02:02:04.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e00838430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.338+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 0x558e00835780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e00838bb0 0x558e00835cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd109ee3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e0083b780 con 0x558e0082d9d0 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd109ee3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e0074ef20 con 0x558e00838bb0 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd109ee3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558e0083b900 con 0x558e0082e2f0 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd109ee3740 10 client.0 _ll_register_callbacks cb 0x558e00718030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1050a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e00838430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1050a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e00838430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34848/0 (socket says 172.21.15.103:34848) 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1050a5640 1 -- 172.21.15.103:0/3880799930 learned_addr learned my addr 172.21.15.103:0/3880799930 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1058a6640 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 0x558e00835780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1050a5640 1 -- 172.21.15.103:0/3880799930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 msgr2=0x558e00835780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1050a5640 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 0x558e00835780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1050a5640 1 -- 172.21.15.103:0/3880799930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e00838bb0 msgr2=0x558e00835cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:04.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1050a5640 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e00838bb0 0x558e00835cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1050a5640 1 -- 172.21.15.103:0/3880799930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e005052b0 con 0x558e0082d9d0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd1050a5640 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e00838430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000ae90 tx=0x7fd0f000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27da8af55a4835ff server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd0e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd0e97fa640 1 -- 172.21.15.103:0/3880799930 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0021060 con 0x558e0082d9d0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd109ee3740 1 -- 172.21.15.103:0/3880799930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0fc0316a0 con 0x558e0082d9d0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd0e97fa640 1 -- 172.21.15.103:0/3880799930 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fd0f001c430 con 0x558e0082d9d0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd0e97fa640 1 -- 172.21.15.103:0/3880799930 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0f0028050 con 0x558e0082d9d0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd109ee3740 10 client.15081 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.339+0000 7fd109ee3740 1 -- 172.21.15.103:0/3880799930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e00847140 con 0x558e0082d9d0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd0e97fa640 1 -- 172.21.15.103:0/3880799930 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0fc0316a0 con 0x558e0082d9d0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd0e97fa640 1 -- 172.21.15.103:0/3880799930 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd0f001a5f0 con 0x558e0082d9d0 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd109ee3740 10 client.15081 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd109ee3740 10 client.15081 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-05T02:02:04.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd109ee3740 20 client.15081 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x558e00849780 0x558e0084abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd109ee3740 4 client.15081 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd109ee3740 1 -- 172.21.15.103:0/3880799930 --> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e007a1600 con 0x558e00849780 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd1058a6640 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x558e00849780 0x558e0084abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd0e97fa640 10 client.15081 ms_handle_connect on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.340+0000 7fd1058a6640 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x558e00849780 0x558e0084abf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=31d8ec1b698708e1 server_cookie=7cc69731b0c812b0 in_seq=0 out_seq=0 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.341+0000 7fd0e97fa640 1 -- 172.21.15.103:0/3880799930 <== mds.4 v2:172.21.15.139:6834/3204357322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e007a1600 con 0x558e00849780 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.341+0000 7fd0e97fa640 10 client.15081 handle_command_reply: tid=0 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.341+0000 7fd109ee3740 1 client.15081 shutdown 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.341+0000 7fd109ee3740 1 -- 172.21.15.103:0/3880799930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 msgr2=0x558e00838430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.341+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e00838430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000ae90 tx=0x7fd0f000f790 comp rx=0 tx=0).stop 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd109ee3740 1 -- 172.21.15.103:0/3880799930 shutdown_connections 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x558e00849780 0x558e0084abf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e00838bb0 0x558e00835cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 0x558e00835780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3880799930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e00838430 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd0e97fa640 0 client.15081 ms_handle_reset on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:04.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd109ee3740 1 -- 172.21.15.103:0/3880799930 >> 172.21.15.103:0/3880799930 conn(0x558e00829f60 msgr2=0x558e0082fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:04.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd109ee3740 1 -- 172.21.15.103:0/3880799930 shutdown_connections 2024-09-05T02:02:04.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd109ee3740 1 -- 172.21.15.103:0/3880799930 wait complete. 2024-09-05T02:02:04.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.342+0000 7fd109ee3740 20 client.15081 trim_cache size 0 max 16384 2024-09-05T02:02:04.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.360+0000 7fd109ee3740 1 Processor -- start 2024-09-05T02:02:04.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.360+0000 7fd109ee3740 1 -- start start 2024-09-05T02:02:04.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.360+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e0084abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.360+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e0082e2f0 0x558e00835750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.360+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e00838bb0 0x558e00835c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.360+0000 7fd109ee3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e00848f30 con 0x558e0082d9d0 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.360+0000 7fd109ee3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e00849600 con 0x558e0082e2f0 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.360+0000 7fd109ee3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558e00849cd0 con 0x558e00838bb0 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd1050a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e0084abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd1050a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e0084abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34864/0 (socket says 172.21.15.103:34864) 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd1050a5640 1 -- 172.21.15.103:0/3086513440 learned_addr learned my addr 172.21.15.103:0/3086513440 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd1050a5640 1 -- 172.21.15.103:0/3086513440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e00838bb0 msgr2=0x558e00835c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd1050a5640 1 --2- 172.21.15.103:0/3086513440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e00838bb0 0x558e00835c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd1050a5640 1 -- 172.21.15.103:0/3086513440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e0082e2f0 msgr2=0x558e00835750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd1050a5640 1 --2- 172.21.15.103:0/3086513440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e0082e2f0 0x558e00835750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd1050a5640 1 -- 172.21.15.103:0/3086513440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e008387a0 con 0x558e0082d9d0 2024-09-05T02:02:04.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd1050a5640 1 --2- 172.21.15.103:0/3086513440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e0084abf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000a690 tx=0x7fd0f000ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d4ab884d417c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:04.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd0eb7fe640 1 -- 172.21.15.103:0/3086513440 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f000a440 con 0x558e0082d9d0 2024-09-05T02:02:04.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd0eb7fe640 1 -- 172.21.15.103:0/3086513440 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0f002c430 con 0x558e0082d9d0 2024-09-05T02:02:04.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.361+0000 7fd0eb7fe640 1 -- 172.21.15.103:0/3086513440 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f001c690 con 0x558e0082d9d0 2024-09-05T02:02:04.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 -- 172.21.15.103:0/3086513440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 msgr2=0x558e0084abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:04.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3086513440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e0084abf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000a690 tx=0x7fd0f000ffc0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 -- 172.21.15.103:0/3086513440 shutdown_connections 2024-09-05T02:02:04.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3086513440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e00838bb0 0x558e00835c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3086513440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e0082e2f0 0x558e00835750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 --2- 172.21.15.103:0/3086513440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0082d9d0 0x558e0084abf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 -- 172.21.15.103:0/3086513440 >> 172.21.15.103:0/3086513440 conn(0x558e00829f60 msgr2=0x558e00835040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 -- 172.21.15.103:0/3086513440 shutdown_connections 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 -- 172.21.15.103:0/3086513440 wait complete. 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 -- start start 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.362+0000 7fd109ee3740 1 Processor -- start 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e0082d9d0 0x558e00853560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 0x558e0084e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd109ee3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e00838bb0 0x558e0084edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd109ee3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e0084aa70 con 0x558e00838bb0 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd109ee3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e0084a3a0 con 0x558e0082d9d0 2024-09-05T02:02:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd109ee3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558e0084b140 con 0x558e0082e2f0 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd109ee3740 10 client.0 _ll_register_callbacks cb 0x7fd0f000acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1050a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e00838bb0 0x558e0084edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1050a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e00838bb0 0x558e0084edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34872/0 (socket says 172.21.15.103:34872) 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1050a5640 1 -- 172.21.15.103:0/582234947 learned_addr learned my addr 172.21.15.103:0/582234947 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1058a6640 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 0x558e0084e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1050a5640 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e0082d9d0 0x558e00853560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1058a6640 1 -- 172.21.15.103:0/582234947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e0082d9d0 msgr2=0x558e00853560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1058a6640 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e0082d9d0 0x558e00853560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1058a6640 1 -- 172.21.15.103:0/582234947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e00838bb0 msgr2=0x558e0084edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1058a6640 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e00838bb0 0x558e0084edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1058a6640 1 -- 172.21.15.103:0/582234947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e0084f5d0 con 0x558e0082e2f0 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.363+0000 7fd1050a5640 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e00838bb0 0x558e0084edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:02:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd1058a6640 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 0x558e0084e890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x558e00836dc0 tx=0x7fd0fc02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffdcb55d669f1e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd0e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd0e97fa640 1 -- 172.21.15.103:0/582234947 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc050060 con 0x558e0082e2f0 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd109ee3740 1 -- 172.21.15.103:0/582234947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f002c430 con 0x558e0082e2f0 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd0e97fa640 1 -- 172.21.15.103:0/582234947 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fd0fc005570 con 0x558e0082e2f0 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd0e97fa640 1 -- 172.21.15.103:0/582234947 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc057050 con 0x558e0082e2f0 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd109ee3740 10 client.24727 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd109ee3740 1 -- 172.21.15.103:0/582234947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e008b3e00 con 0x558e0082e2f0 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd0e97fa640 1 -- 172.21.15.103:0/582234947 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0f002c430 con 0x558e0082e2f0 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd0e97fa640 1 -- 172.21.15.103:0/582234947 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd0fc0037f0 con 0x558e0082e2f0 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd109ee3740 10 client.24727 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd109ee3740 10 client.24727 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-05T02:02:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd109ee3740 20 client.24727 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd109ee3740 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x558e008b85a0 0x558e008b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd109ee3740 4 client.24727 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.364+0000 7fd109ee3740 1 -- 172.21.15.103:0/582234947 --> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] -- command(tid 0: {"prefix": "status"}) -- 0x558e0079bb10 con 0x558e008b85a0 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.365+0000 7fd1050a5640 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x558e008b85a0 0x558e008b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.365+0000 7fd0e97fa640 10 client.24727 ms_handle_connect on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.365+0000 7fd1050a5640 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x558e008b85a0 0x558e008b72b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d725b4927c509706 server_cookie=7875c7629714cdc7 in_seq=0 out_seq=0 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.365+0000 7fd0e97fa640 1 -- 172.21.15.103:0/582234947 <== mds.4 v2:172.21.15.139:6834/3204357322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558e0079bb10 con 0x558e008b85a0 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.365+0000 7fd0e97fa640 10 client.24727 handle_command_reply: tid=0 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.365+0000 7fd109ee3740 1 client.24727 shutdown 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 -- 172.21.15.103:0/582234947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 msgr2=0x558e0084e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 0x558e0084e890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x558e00836dc0 tx=0x7fd0fc02a3f0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 -- 172.21.15.103:0/582234947 shutdown_connections 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e00838bb0 0x558e0084edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0082e2f0 0x558e0084e890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x558e008b85a0 0x558e008b72b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 --2- 172.21.15.103:0/582234947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e0082d9d0 0x558e00853560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd0e97fa640 0 client.24727 ms_handle_reset on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 -- 172.21.15.103:0/582234947 >> 172.21.15.103:0/582234947 conn(0x558e00829f60 msgr2=0x558e00835860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 -- 172.21.15.103:0/582234947 shutdown_connections 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 1 -- 172.21.15.103:0/582234947 wait complete. 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd109ee3740 20 client.24727 trim_cache size 0 max 16384 2024-09-05T02:02:04.374 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24449, 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 66.146008804999994, 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 88, 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 141.876826658 2024-09-05T02:02:04.375 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd107b0a640 1 -- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0ec0a2c10 msgr2=0x7fd0ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd107b0a640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd0f809ba00 tx=0x7fd0f806a9f0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd107b0a640 1 -- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100106f20 msgr2=0x7fd10019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.366+0000 7fd107b0a640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100106f20 0x7fd10019cca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800eaa0 tx=0x7fd0f800eea0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.367+0000 7fd107b0a640 1 -- 172.21.15.103:0/63755839 shutdown_connections 2024-09-05T02:02:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.367+0000 7fd107b0a640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd100106f20 0x7fd10019cca0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.367+0000 7fd107b0a640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.367+0000 7fd107b0a640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1001027a0 0x7fd100198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.367+0000 7fd107b0a640 1 --2- 172.21.15.103:0/63755839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd100101df0 0x7fd1001983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.367+0000 7fd107b0a640 1 -- 172.21.15.103:0/63755839 >> 172.21.15.103:0/63755839 conn(0x7fd1000fd8b0 msgr2=0x7fd1000febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.367+0000 7fd107b0a640 1 -- 172.21.15.103:0/63755839 shutdown_connections 2024-09-05T02:02:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:04.367+0000 7fd107b0a640 1 -- 172.21.15.103:0/63755839 wait complete. 2024-09-05T02:02:04.547 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:04 smithi139 ceph-mon[34262]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:04 smithi167 ceph-mon[35122]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:04.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:04 smithi103 ceph-mon[30902]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:05.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.c status 2024-09-05T02:02:05.486 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:06.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.983+0000 7f80b4b02640 1 -- 172.21.15.103:0/1528103522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0100e20 msgr2=0x7f80b0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:06.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.983+0000 7f80b4b02640 1 --2- 172.21.15.103:0/1528103522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0100e20 0x7f80b0101200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009900 tx=0x7f80a002a480 comp rx=0 tx=0).stop 2024-09-05T02:02:06.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.983+0000 7f80b4b02640 1 -- 172.21.15.103:0/1528103522 shutdown_connections 2024-09-05T02:02:06.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.983+0000 7f80b4b02640 1 --2- 172.21.15.103:0/1528103522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b0102ef0 0x7f80b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.983+0000 7f80b4b02640 1 --2- 172.21.15.103:0/1528103522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b0102570 0x7f80b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.983+0000 7f80b4b02640 1 --2- 172.21.15.103:0/1528103522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0100e20 0x7f80b0101200 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.983+0000 7f80b4b02640 1 -- 172.21.15.103:0/1528103522 >> 172.21.15.103:0/1528103522 conn(0x7f80b00fc7d0 msgr2=0x7f80b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:06.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.983+0000 7f80b4b02640 1 -- 172.21.15.103:0/1528103522 shutdown_connections 2024-09-05T02:02:06.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.983+0000 7f80b4b02640 1 -- 172.21.15.103:0/1528103522 wait complete. 2024-09-05T02:02:06.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.984+0000 7f80b4b02640 1 Processor -- start 2024-09-05T02:02:06.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.984+0000 7f80b4b02640 1 -- start start 2024-09-05T02:02:06.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.984+0000 7f80b4b02640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b0100e20 0x7f80b019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.984+0000 7f80b4b02640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b0102570 0x7f80b019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.984+0000 7f80b4b02640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0102ef0 0x7f80b0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.984+0000 7f80b4b02640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80b0110e50 con 0x7f80b0102ef0 2024-09-05T02:02:06.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80b4b02640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f80b0110cd0 con 0x7f80b0102570 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80b4b02640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80b0110fd0 con 0x7f80b0100e20 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80aed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0102ef0 0x7f80b0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80aed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0102ef0 0x7f80b0198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34894/0 (socket says 172.21.15.103:34894) 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80aed9d640 1 -- 172.21.15.103:0/3242057226 learned_addr learned my addr 172.21.15.103:0/3242057226 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80ae59c640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b0100e20 0x7f80b019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80add9b640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b0102570 0x7f80b019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80aed9d640 1 -- 172.21.15.103:0/3242057226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b0100e20 msgr2=0x7f80b019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80aed9d640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b0100e20 0x7f80b019d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80aed9d640 1 -- 172.21.15.103:0/3242057226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b0102570 msgr2=0x7f80b019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80aed9d640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b0102570 0x7f80b019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80aed9d640 1 -- 172.21.15.103:0/3242057226 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b0198cb0 con 0x7f80b0102ef0 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80aed9d640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0102ef0 0x7f80b0198530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f80a800cad0 tx=0x7f80a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f809f7fe640 1 -- 172.21.15.103:0/3242057226 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a800ae20 con 0x7f80b0102ef0 2024-09-05T02:02:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f809f7fe640 1 -- 172.21.15.103:0/3242057226 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80a8005020 con 0x7f80b0102ef0 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f809f7fe640 1 -- 172.21.15.103:0/3242057226 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a800f6b0 con 0x7f80b0102ef0 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80b4b02640 1 -- 172.21.15.103:0/3242057226 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b0198fa0 con 0x7f80b0102ef0 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.985+0000 7f80b4b02640 1 -- 172.21.15.103:0/3242057226 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b01a4fe0 con 0x7f80b0102ef0 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.987+0000 7f809f7fe640 1 -- 172.21.15.103:0/3242057226 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f80a80051d0 con 0x7f80b0102ef0 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80b6edb740 1 Processor -- start 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80b6edb740 1 -- start start 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8791f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b879201670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b879201bb0 0x55b879203fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80b6edb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b87915d090 con 0x55b8791f69d0 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80b6edb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b879104a70 con 0x55b8791f72f0 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80b6edb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b879159050 con 0x55b879201bb0 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80add9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8791f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80add9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8791f6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34908/0 (socket says 172.21.15.103:34908) 2024-09-05T02:02:06.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80add9b640 1 -- 172.21.15.103:0/2695200631 learned_addr learned my addr 172.21.15.103:0/2695200631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.988+0000 7f80ae59c640 1 --2- 172.21.15.103:0/2695200631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b879201670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f80add9b640 1 -- 172.21.15.103:0/2695200631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b879201bb0 msgr2=0x55b879203fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f80add9b640 1 --2- 172.21.15.103:0/2695200631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b879201bb0 0x55b879203fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f80add9b640 1 -- 172.21.15.103:0/2695200631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 msgr2=0x55b879201670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f80add9b640 1 --2- 172.21.15.103:0/2695200631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b879201670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f809f7fe640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f80980a2c10 0x7f80980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f80add9b640 1 -- 172.21.15.103:0/2695200631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b879111130 con 0x55b8791f69d0 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f80add9b640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f80980a2c10 0x7f80980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f809f7fe640 1 -- 172.21.15.103:0/3242057226 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f80a80cf610 con 0x7f80b0102ef0 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f80add9b640 1 --2- 172.21.15.103:0/2695200631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8791f6db0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f80a4006e10 tx=0x7f80a4009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01dfba1e071b6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f809cff9640 1 -- 172.21.15.103:0/2695200631 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4014630 con 0x55b8791f69d0 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f809cff9640 1 -- 172.21.15.103:0/2695200631 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80a4010040 con 0x55b8791f69d0 2024-09-05T02:02:06.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.989+0000 7f809cff9640 1 -- 172.21.15.103:0/2695200631 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4020720 con 0x55b8791f69d0 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 -- 172.21.15.103:0/2695200631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 msgr2=0x55b8791f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80add9b640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f80980a2c10 0x7f80980a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80a401d000 tx=0x7f80a4020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 --2- 172.21.15.103:0/2695200631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8791f6db0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f80a4006e10 tx=0x7f80a4009ea0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 -- 172.21.15.103:0/2695200631 shutdown_connections 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 --2- 172.21.15.103:0/2695200631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b879201bb0 0x55b879203fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 --2- 172.21.15.103:0/2695200631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b879201670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 --2- 172.21.15.103:0/2695200631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8791f6db0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 -- 172.21.15.103:0/2695200631 >> 172.21.15.103:0/2695200631 conn(0x55b8791f2f60 msgr2=0x55b8791f3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 -- 172.21.15.103:0/2695200631 shutdown_connections 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 -- 172.21.15.103:0/2695200631 wait complete. 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 -- start start 2024-09-05T02:02:06.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.990+0000 7f80b6edb740 1 Processor -- start 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8792014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b8791fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b879201bb0 0x55b8791fed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80b6edb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b879204780 con 0x55b8791f69d0 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80b6edb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b879117f20 con 0x55b8791f72f0 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80b6edb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b879204900 con 0x55b879201bb0 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8792014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80b6edb740 10 client.0 _ll_register_callbacks cb 0x55b8790e1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8792014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34916/0 (socket says 172.21.15.103:34916) 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80ae59c640 1 -- 172.21.15.103:0/4032490268 learned_addr learned my addr 172.21.15.103:0/4032490268 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80add9b640 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b8791fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80ae59c640 1 -- 172.21.15.103:0/4032490268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b879201bb0 msgr2=0x55b8791fed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80ae59c640 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b879201bb0 0x55b8791fed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80ae59c640 1 -- 172.21.15.103:0/4032490268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 msgr2=0x55b8791fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80ae59c640 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b8791fe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80ae59c640 1 -- 172.21.15.103:0/4032490268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b878ece2b0 con 0x55b8791f69d0 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80ae59c640 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8792014f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f80a002a9a0 tx=0x7f80a0031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cbd4fcf2ec6fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f8095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f8095ffb640 1 -- 172.21.15.103:0/4032490268 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0035040 con 0x55b8791f69d0 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f80b6edb740 1 -- 172.21.15.103:0/4032490268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a4010040 con 0x55b8791f69d0 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.991+0000 7f8095ffb640 1 -- 172.21.15.103:0/4032490268 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f80a0004250 con 0x55b8791f69d0 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f8095ffb640 1 -- 172.21.15.103:0/4032490268 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f80a0004450 con 0x55b8791f69d0 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f80b6edb740 10 client.15099 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f80b6edb740 1 -- 172.21.15.103:0/4032490268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b879210180 con 0x55b8791f69d0 2024-09-05T02:02:06.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f8095ffb640 1 -- 172.21.15.103:0/4032490268 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80a4010040 con 0x55b8791f69d0 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f8095ffb640 1 -- 172.21.15.103:0/4032490268 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f80a0044070 con 0x55b8791f69d0 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f80b6edb740 10 client.15099 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f80b6edb740 10 client.15099 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f80b6edb740 20 client.15099 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f80b6edb740 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x55b8792127f0 0x55b879213c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f80b6edb740 4 client.15099 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f80b6edb740 1 -- 172.21.15.103:0/4032490268 --> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b87916a600 con 0x55b8792127f0 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.992+0000 7f80add9b640 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x55b8792127f0 0x55b879213c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.993+0000 7f8095ffb640 10 client.15099 ms_handle_connect on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.993+0000 7f80add9b640 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x55b8792127f0 0x55b879213c60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23add92b46626f53 server_cookie=2885d59c32ff3ff5 in_seq=0 out_seq=0 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f8095ffb640 1 -- 172.21.15.103:0/4032490268 <== mds.3 v2:172.21.15.167:6836/3468047781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b87916a600 con 0x55b8792127f0 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f8095ffb640 10 client.15099 handle_command_reply: tid=0 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 client.15099 shutdown 2024-09-05T02:02:06.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 -- 172.21.15.103:0/4032490268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 msgr2=0x55b8792014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8792014f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f80a002a9a0 tx=0x7f80a0031660 comp rx=0 tx=0).stop 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 -- 172.21.15.103:0/4032490268 shutdown_connections 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x55b8792127f0 0x55b879213c60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b879201bb0 0x55b8791fed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f8095ffb640 0 client.15099 ms_handle_reset on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b8791fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 --2- 172.21.15.103:0/4032490268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b8792014f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 -- 172.21.15.103:0/4032490268 >> 172.21.15.103:0/4032490268 conn(0x55b8791f2f60 msgr2=0x55b8791f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 -- 172.21.15.103:0/4032490268 shutdown_connections 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 1 -- 172.21.15.103:0/4032490268 wait complete. 2024-09-05T02:02:06.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:05.994+0000 7f80b6edb740 20 client.15099 trim_cache size 0 max 16384 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.011+0000 7f80b6edb740 1 Processor -- start 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80b6edb740 1 -- start start 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f69d0 0x55b8791f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b8792006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b879201bb0 0x55b8791fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80b6edb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b879211fa0 con 0x55b879201bb0 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80b6edb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b879212670 con 0x55b8791f72f0 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80b6edb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b879212d40 con 0x55b8791f69d0 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80aed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b879201bb0 0x55b8791fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80aed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b879201bb0 0x55b8791fbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34924/0 (socket says 172.21.15.103:34924) 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80aed9d640 1 -- 172.21.15.103:0/3483221444 learned_addr learned my addr 172.21.15.103:0/3483221444 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80add9b640 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b8792006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80ae59c640 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f69d0 0x55b8791f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80aed9d640 1 -- 172.21.15.103:0/3483221444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f69d0 msgr2=0x55b8791f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80aed9d640 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f69d0 0x55b8791f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80aed9d640 1 -- 172.21.15.103:0/3483221444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 msgr2=0x55b8792006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80aed9d640 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b8792006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80aed9d640 1 -- 172.21.15.103:0/3483221444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b879201860 con 0x55b879201bb0 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80ae59c640 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f69d0 0x55b8791f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.012+0000 7f80aed9d640 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b879201bb0 0x55b8791fbac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f80a8097e10 tx=0x7f80a809b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac1eea1992c4e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f8097fff640 1 -- 172.21.15.103:0/3483221444 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a801d040 con 0x55b879201bb0 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f8097fff640 1 -- 172.21.15.103:0/3483221444 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80a8005560 con 0x55b879201bb0 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f8097fff640 1 -- 172.21.15.103:0/3483221444 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8097700 con 0x55b879201bb0 2024-09-05T02:02:06.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f80b6edb740 1 -- 172.21.15.103:0/3483221444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b879201bb0 msgr2=0x55b8791fbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f80b6edb740 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b879201bb0 0x55b8791fbac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f80a8097e10 tx=0x7f80a809b440 comp rx=0 tx=0).stop 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f80b6edb740 1 -- 172.21.15.103:0/3483221444 shutdown_connections 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f80b6edb740 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b879201bb0 0x55b8791fbac0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f80b6edb740 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b8791f72f0 0x55b8792006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f80b6edb740 1 --2- 172.21.15.103:0/3483221444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f69d0 0x55b8791f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f80b6edb740 1 -- 172.21.15.103:0/3483221444 >> 172.21.15.103:0/3483221444 conn(0x55b8791f2f60 msgr2=0x55b8791fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f80b6edb740 1 -- 172.21.15.103:0/3483221444 shutdown_connections 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.013+0000 7f80b6edb740 1 -- 172.21.15.103:0/3483221444 wait complete. 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80b6edb740 1 -- start start 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80b6edb740 1 Processor -- start 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b87921c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f72f0 0x55b879277700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80b6edb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b879201bb0 0x55b879277c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80b6edb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b879213ae0 con 0x55b8791f69d0 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80b6edb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b879213410 con 0x55b879201bb0 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80b6edb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8792141b0 con 0x55b8791f72f0 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80b6edb740 10 client.0 _ll_register_callbacks cb 0x7f80a0032ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b87921c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b87921c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34932/0 (socket says 172.21.15.103:34932) 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80ae59c640 1 -- 172.21.15.103:0/1915113855 learned_addr learned my addr 172.21.15.103:0/1915113855 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80add9b640 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f72f0 0x55b879277700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80aed9d640 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b879201bb0 0x55b879277c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80ae59c640 1 -- 172.21.15.103:0/1915113855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f72f0 msgr2=0x55b879277700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80ae59c640 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f72f0 0x55b879277700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80ae59c640 1 -- 172.21.15.103:0/1915113855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b879201bb0 msgr2=0x55b879277c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80ae59c640 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b879201bb0 0x55b879277c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.014+0000 7f80ae59c640 1 -- 172.21.15.103:0/1915113855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b879278180 con 0x55b8791f69d0 2024-09-05T02:02:06.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80ae59c640 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b87921c3a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f80a0005ec0 tx=0x7f80a0046ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f0c4239ff83d90 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f8095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f8095ffb640 1 -- 172.21.15.103:0/1915113855 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0035070 con 0x55b8791f69d0 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f8095ffb640 1 -- 172.21.15.103:0/1915113855 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f80a00318e0 con 0x55b8791f69d0 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80b6edb740 1 -- 172.21.15.103:0/1915113855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a8005560 con 0x55b8791f69d0 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f8095ffb640 1 -- 172.21.15.103:0/1915113855 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f80a0031b00 con 0x55b8791f69d0 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80b6edb740 10 client.15105 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80b6edb740 1 -- 172.21.15.103:0/1915113855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b879219bb0 con 0x55b8791f69d0 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f8095ffb640 1 -- 172.21.15.103:0/1915113855 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80a8005560 con 0x55b8791f69d0 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f8095ffb640 1 -- 172.21.15.103:0/1915113855 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f80a0044030 con 0x55b8791f69d0 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80b6edb740 10 client.15105 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80b6edb740 10 client.15105 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80b6edb740 20 client.15105 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80b6edb740 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x55b879285900 0x55b879284610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:06.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80b6edb740 4 client.15105 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.015+0000 7f80b6edb740 1 -- 172.21.15.103:0/1915113855 --> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] -- command(tid 0: {"prefix": "status"}) -- 0x55b879164b10 con 0x55b879285900 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80add9b640 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x55b879285900 0x55b879284610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80add9b640 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x55b879285900 0x55b879284610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cba1eba6b867b2fe server_cookie=b6d1145555da69ee in_seq=0 out_seq=0 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f8095ffb640 10 client.15105 ms_handle_connect on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f8095ffb640 1 -- 172.21.15.103:0/1915113855 <== mds.3 v2:172.21.15.167:6836/3468047781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b879164b10 con 0x55b879285900 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f8095ffb640 10 client.15105 handle_command_reply: tid=0 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80b6edb740 1 client.15105 shutdown 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80b6edb740 1 -- 172.21.15.103:0/1915113855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 msgr2=0x55b87921c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80b6edb740 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b87921c3a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f80a0005ec0 tx=0x7f80a0046ca0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80b6edb740 1 -- 172.21.15.103:0/1915113855 shutdown_connections 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80b6edb740 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x55b879285900 0x55b879284610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80b6edb740 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b879201bb0 0x55b879277c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80b6edb740 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8791f72f0 0x55b879277700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80b6edb740 1 --2- 172.21.15.103:0/1915113855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8791f69d0 0x55b87921c3a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f80b6edb740 1 -- 172.21.15.103:0/1915113855 >> 172.21.15.103:0/1915113855 conn(0x55b8791f2f60 msgr2=0x55b8791f2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.016+0000 7f8095ffb640 0 client.15105 ms_handle_reset on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b6edb740 1 -- 172.21.15.103:0/1915113855 shutdown_connections 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b6edb740 1 -- 172.21.15.103:0/1915113855 wait complete. 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b6edb740 20 client.15105 trim_cache size 0 max 16384 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24457, 2024-09-05T02:02:06.035 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:06.036 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:06.036 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:06.036 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 68.809819417, 2024-09-05T02:02:06.036 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:06.036 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 88, 2024-09-05T02:02:06.036 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:06.036 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 137.77424623499999 2024-09-05T02:02:06.036 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 -- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f80980a2c10 msgr2=0x7f80980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f80980a2c10 0x7f80980a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80a401d000 tx=0x7f80a4020000 comp rx=0 tx=0).stop 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 -- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0102ef0 msgr2=0x7f80b0198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0102ef0 0x7f80b0198530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f80a800cad0 tx=0x7f80a800cf90 comp rx=0 tx=0).stop 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 -- 172.21.15.103:0/3242057226 shutdown_connections 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80b0102ef0 0x7f80b0198530 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b0102570 0x7f80b019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f80980a2c10 0x7f80980a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 --2- 172.21.15.103:0/3242057226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b0100e20 0x7f80b019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:06.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 -- 172.21.15.103:0/3242057226 >> 172.21.15.103:0/3242057226 conn(0x7f80b00fc7d0 msgr2=0x7f80b01039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:06.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 -- 172.21.15.103:0/3242057226 shutdown_connections 2024-09-05T02:02:06.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:06.017+0000 7f80b4b02640 1 -- 172.21.15.103:0/3242057226 wait complete. 2024-09-05T02:02:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:06 smithi167 ceph-mon[35122]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:02:06.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:06 smithi139 ceph-mon[34262]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:02:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:06 smithi103 ceph-mon[30902]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:02:06.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T02:02:07.060 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:07.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.529+0000 7f604c500640 1 -- 172.21.15.103:0/170116265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 msgr2=0x7f6044103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:07.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.529+0000 7f604c500640 1 --2- 172.21.15.103:0/170116265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 0x7f6044103120 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6034009900 tx=0x7f603402a480 comp rx=0 tx=0).stop 2024-09-05T02:02:07.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.529+0000 7f604c500640 1 -- 172.21.15.103:0/170116265 shutdown_connections 2024-09-05T02:02:07.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.529+0000 7f604c500640 1 --2- 172.21.15.103:0/170116265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f604410c170 0x7f604410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.529+0000 7f604c500640 1 --2- 172.21.15.103:0/170116265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6044103660 0x7f604410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.529+0000 7f604c500640 1 --2- 172.21.15.103:0/170116265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 0x7f6044103120 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.529+0000 7f604c500640 1 -- 172.21.15.103:0/170116265 >> 172.21.15.103:0/170116265 conn(0x7f60440fc7a0 msgr2=0x7f60440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.529+0000 7f604c500640 1 -- 172.21.15.103:0/170116265 shutdown_connections 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.529+0000 7f604c500640 1 -- 172.21.15.103:0/170116265 wait complete. 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604c500640 1 Processor -- start 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604c500640 1 -- start start 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604c500640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 0x7f60441009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604c500640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6044103660 0x7f6044100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604c500640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f604410c170 0x7f6044101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604c500640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6044111e20 con 0x7f6044102d40 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604c500640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6044111ca0 con 0x7f604410c170 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604c500640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6044111fa0 con 0x7f6044103660 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604a29c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 0x7f60441009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604a29c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 0x7f60441009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34956/0 (socket says 172.21.15.103:34956) 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604a29c640 1 -- 172.21.15.103:0/3586055960 learned_addr learned my addr 172.21.15.103:0/3586055960 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604aa9d640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f604410c170 0x7f6044101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f6049a9b640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6044103660 0x7f6044100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604a29c640 1 -- 172.21.15.103:0/3586055960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6044103660 msgr2=0x7f6044100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604a29c640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6044103660 0x7f6044100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604a29c640 1 -- 172.21.15.103:0/3586055960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f604410c170 msgr2=0x7f6044101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604a29c640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f604410c170 0x7f6044101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604a29c640 1 -- 172.21.15.103:0/3586055960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60441a6810 con 0x7f6044102d40 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f6049a9b640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6044103660 0x7f6044100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.530+0000 7f604aa9d640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f604410c170 0x7f6044101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.531+0000 7f604a29c640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 0x7f60441009a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6034005940 tx=0x7f6034004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.531+0000 7f603b7fe640 1 -- 172.21.15.103:0/3586055960 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603403f030 con 0x7f6044102d40 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.531+0000 7f604c500640 1 -- 172.21.15.103:0/3586055960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60441a6aa0 con 0x7f6044102d40 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.531+0000 7f603b7fe640 1 -- 172.21.15.103:0/3586055960 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6034003aa0 con 0x7f6044102d40 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.531+0000 7f603b7fe640 1 -- 172.21.15.103:0/3586055960 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034035070 con 0x7f6044102d40 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.531+0000 7f604c500640 1 -- 172.21.15.103:0/3586055960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60441a6fd0 con 0x7f6044102d40 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f603b7fe640 1 -- 172.21.15.103:0/3586055960 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6034042030 con 0x7f6044102d40 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604e8d9740 1 Processor -- start 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604e8d9740 1 -- start start 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b5c9d0 0x559747b5cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b67670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b67bb0 0x559747b69fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604e8d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559747ac3090 con 0x559747b67bb0 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604e8d9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559747a6aa70 con 0x559747b5d2f0 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604e8d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559747abf050 con 0x559747b5c9d0 2024-09-05T02:02:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f6049a9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b67bb0 0x559747b69fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f6049a9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b67bb0 0x559747b69fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34968/0 (socket says 172.21.15.103:34968) 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f6049a9b640 1 -- 172.21.15.103:0/849714198 learned_addr learned my addr 172.21.15.103:0/849714198 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604a29c640 1 --2- 172.21.15.103:0/849714198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b67670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.534+0000 7f604aa9d640 1 --2- 172.21.15.103:0/849714198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b5c9d0 0x559747b5cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f6049a9b640 1 -- 172.21.15.103:0/849714198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b5c9d0 msgr2=0x559747b5cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f6049a9b640 1 --2- 172.21.15.103:0/849714198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b5c9d0 0x559747b5cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f6049a9b640 1 -- 172.21.15.103:0/849714198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 msgr2=0x559747b67670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f6049a9b640 1 --2- 172.21.15.103:0/849714198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b67670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f603b7fe640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60300a2c10 0x7f60300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f6049a9b640 1 -- 172.21.15.103:0/849714198 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559747a77130 con 0x559747b67bb0 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f604aa9d640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60300a2c10 0x7f60300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f6049a9b640 1 --2- 172.21.15.103:0/849714198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b67bb0 0x559747b69fa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f604000bca0 tx=0x7f604000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e19bbc85ec4856 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f603b7fe640 1 -- 172.21.15.103:0/3586055960 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f60340efd90 con 0x7f6044102d40 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f6038ff9640 1 -- 172.21.15.103:0/849714198 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040011440 con 0x559747b67bb0 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f6038ff9640 1 -- 172.21.15.103:0/849714198 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6040011a90 con 0x559747b67bb0 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.535+0000 7f6038ff9640 1 -- 172.21.15.103:0/849714198 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604001b9d0 con 0x559747b67bb0 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 -- 172.21.15.103:0/849714198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b67bb0 msgr2=0x559747b69fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604aa9d640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60300a2c10 0x7f60300a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f603c005fd0 tx=0x7f603c009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 --2- 172.21.15.103:0/849714198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b67bb0 0x559747b69fa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f604000bca0 tx=0x7f604000f730 comp rx=0 tx=0).stop 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 -- 172.21.15.103:0/849714198 shutdown_connections 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 --2- 172.21.15.103:0/849714198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b67bb0 0x559747b69fa0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 --2- 172.21.15.103:0/849714198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b67670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 --2- 172.21.15.103:0/849714198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b5c9d0 0x559747b5cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 -- 172.21.15.103:0/849714198 >> 172.21.15.103:0/849714198 conn(0x559747b58f60 msgr2=0x559747b59340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 -- 172.21.15.103:0/849714198 shutdown_connections 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 -- 172.21.15.103:0/849714198 wait complete. 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 -- start start 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.536+0000 7f604e8d9740 1 Processor -- start 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b67400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b64750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747b64c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604e8d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559747b6a780 con 0x559747b5c9d0 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604e8d9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559747a7df20 con 0x559747b5d2f0 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604e8d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559747b6a900 con 0x559747b67bb0 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604e8d9740 10 client.0 _ll_register_callbacks cb 0x559747a47030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604a29c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b67400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604a29c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b67400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34972/0 (socket says 172.21.15.103:34972) 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604a29c640 1 -- 172.21.15.103:0/1139487192 learned_addr learned my addr 172.21.15.103:0/1139487192 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f6049a9b640 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b64750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604aa9d640 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747b64c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604a29c640 1 -- 172.21.15.103:0/1139487192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 msgr2=0x559747b64c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604a29c640 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747b64c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604a29c640 1 -- 172.21.15.103:0/1139487192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 msgr2=0x559747b64750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604a29c640 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b64750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604a29c640 1 -- 172.21.15.103:0/1139487192 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597478342b0 con 0x559747b5c9d0 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604aa9d640 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747b64c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604a29c640 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b67400 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60340bc0c0 tx=0x7f6034009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b555417d0c85e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f602dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f602dffb640 1 -- 172.21.15.103:0/1139487192 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034049050 con 0x559747b5c9d0 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f602dffb640 1 -- 172.21.15.103:0/1139487192 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f6034003c50 con 0x559747b5c9d0 2024-09-05T02:02:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f604e8d9740 1 -- 172.21.15.103:0/1139487192 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6040011a90 con 0x559747b5c9d0 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.537+0000 7f602dffb640 1 -- 172.21.15.103:0/1139487192 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60340bbc70 con 0x559747b5c9d0 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f604e8d9740 10 client.15117 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f604e8d9740 1 -- 172.21.15.103:0/1139487192 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559747b76080 con 0x559747b5c9d0 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f602dffb640 1 -- 172.21.15.103:0/1139487192 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6040011a90 con 0x559747b5c9d0 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f602dffb640 1 -- 172.21.15.103:0/1139487192 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f60340dc0b0 con 0x559747b5c9d0 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f604e8d9740 10 client.15117 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f604e8d9740 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f604e8d9740 20 client.15117 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x559747b78660 0x559747b79ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f604e8d9740 4 client.15117 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f604e8d9740 1 -- 172.21.15.103:0/1139487192 --> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559747ad0600 con 0x559747b78660 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.538+0000 7f6049a9b640 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x559747b78660 0x559747b79ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.539+0000 7f6049a9b640 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x559747b78660 0x559747b79ad0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=125b68d91513ae35 server_cookie=21ace68f35f6782d in_seq=0 out_seq=0 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.539+0000 7f602dffb640 10 client.15117 ms_handle_connect on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.539+0000 7f602dffb640 1 -- 172.21.15.103:0/1139487192 <== mds.2 v2:172.21.15.139:6836/2336570561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559747ad0600 con 0x559747b78660 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.539+0000 7f602dffb640 10 client.15117 handle_command_reply: tid=0 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.539+0000 7f604e8d9740 1 client.15117 shutdown 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 -- 172.21.15.103:0/1139487192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 msgr2=0x559747b67400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b67400 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60340bc0c0 tx=0x7f6034009560 comp rx=0 tx=0).stop 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 -- 172.21.15.103:0/1139487192 shutdown_connections 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x559747b78660 0x559747b79ad0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747b64c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b64750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1139487192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b67400 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f602dffb640 0 client.15117 ms_handle_reset on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 -- 172.21.15.103:0/1139487192 >> 172.21.15.103:0/1139487192 conn(0x559747b58f60 msgr2=0x559747b5ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 -- 172.21.15.103:0/1139487192 shutdown_connections 2024-09-05T02:02:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 1 -- 172.21.15.103:0/1139487192 wait complete. 2024-09-05T02:02:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.540+0000 7f604e8d9740 20 client.15117 trim_cache size 0 max 16384 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.557+0000 7f604e8d9740 1 Processor -- start 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.557+0000 7f604e8d9740 1 -- start start 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b77f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b79ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747b64710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604e8d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559747b77e10 con 0x559747b5c9d0 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604e8d9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559747b784e0 con 0x559747b5d2f0 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604e8d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559747b78bb0 con 0x559747b67bb0 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604a29c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b77f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604a29c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b77f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34986/0 (socket says 172.21.15.103:34986) 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604a29c640 1 -- 172.21.15.103:0/1200521612 learned_addr learned my addr 172.21.15.103:0/1200521612 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604a29c640 1 -- 172.21.15.103:0/1200521612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 msgr2=0x559747b64710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604a29c640 1 --2- 172.21.15.103:0/1200521612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747b64710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604a29c640 1 -- 172.21.15.103:0/1200521612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 msgr2=0x559747b79ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604a29c640 1 --2- 172.21.15.103:0/1200521612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b79ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604a29c640 1 -- 172.21.15.103:0/1200521612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559747b67770 con 0x559747b5c9d0 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f604a29c640 1 --2- 172.21.15.103:0/1200521612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b77f90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6034006010 tx=0x7f60340c0660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442f3df4625d1111 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f602ffff640 1 -- 172.21.15.103:0/1200521612 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034049070 con 0x559747b5c9d0 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f602ffff640 1 -- 172.21.15.103:0/1200521612 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f603403b460 con 0x559747b5c9d0 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.558+0000 7f602ffff640 1 -- 172.21.15.103:0/1200521612 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340bba90 con 0x559747b5c9d0 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 -- 172.21.15.103:0/1200521612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 msgr2=0x559747b77f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1200521612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b77f90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6034006010 tx=0x7f60340c0660 comp rx=0 tx=0).stop 2024-09-05T02:02:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 -- 172.21.15.103:0/1200521612 shutdown_connections 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1200521612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747b64710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1200521612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b79ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1200521612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b77f90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 -- 172.21.15.103:0/1200521612 >> 172.21.15.103:0/1200521612 conn(0x559747b58f60 msgr2=0x559747b53ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 -- 172.21.15.103:0/1200521612 shutdown_connections 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 -- 172.21.15.103:0/1200521612 wait complete. 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 -- start start 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 Processor -- start 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.559+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b7d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b7d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604e8d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747be0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604e8d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559747b79950 con 0x559747b5c9d0 2024-09-05T02:02:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604e8d9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559747b79280 con 0x559747b5d2f0 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604e8d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559747b7a020 con 0x559747b67bb0 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604e8d9740 10 client.0 _ll_register_callbacks cb 0x7f6034005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604a29c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b7d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604a29c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b7d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34990/0 (socket says 172.21.15.103:34990) 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604a29c640 1 -- 172.21.15.103:0/1793388495 learned_addr learned my addr 172.21.15.103:0/1793388495 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f6049a9b640 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b7d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604aa9d640 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747be0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604a29c640 1 -- 172.21.15.103:0/1793388495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 msgr2=0x559747be0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604a29c640 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747be0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604a29c640 1 -- 172.21.15.103:0/1793388495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 msgr2=0x559747b7d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604a29c640 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b7d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604a29c640 1 -- 172.21.15.103:0/1793388495 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559747be12a0 con 0x559747b5c9d0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604aa9d640 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747be0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604a29c640 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b7d180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f60340bc0c0 tx=0x7f60340ba640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca80c5646e3edcc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f602dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f602dffb640 1 -- 172.21.15.103:0/1793388495 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034049050 con 0x559747b5c9d0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f602dffb640 1 -- 172.21.15.103:0/1793388495 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f60340f3570 con 0x559747b5c9d0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604e8d9740 1 -- 172.21.15.103:0/1793388495 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559747acab10 con 0x559747b5c9d0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f602dffb640 1 -- 172.21.15.103:0/1793388495 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60340bba90 con 0x559747b5c9d0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604e8d9740 10 client.15123 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.560+0000 7f604e8d9740 1 -- 172.21.15.103:0/1793388495 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559747be12a0 con 0x559747b5c9d0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.561+0000 7f602dffb640 1 -- 172.21.15.103:0/1793388495 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559747acab10 con 0x559747b5c9d0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.561+0000 7f602dffb640 1 -- 172.21.15.103:0/1793388495 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f60340c4910 con 0x559747b5c9d0 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.561+0000 7f604e8d9740 10 client.15123 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.561+0000 7f604e8d9740 10 client.15123 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T02:02:07.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.561+0000 7f604e8d9740 20 client.15123 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.561+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x559747be67a0 0x559747be54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.561+0000 7f604e8d9740 4 client.15123 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.561+0000 7f604e8d9740 1 -- 172.21.15.103:0/1793388495 --> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] -- command(tid 0: {"prefix": "status"}) -- 0x559747a470e0 con 0x559747be67a0 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.561+0000 7f6049a9b640 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x559747be67a0 0x559747be54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f6049a9b640 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x559747be67a0 0x559747be54b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6acfbd792e0365c8 server_cookie=a0da2fa225745d03 in_seq=0 out_seq=0 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f602dffb640 10 client.15123 ms_handle_connect on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f602dffb640 1 -- 172.21.15.103:0/1793388495 <== mds.2 v2:172.21.15.139:6836/2336570561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559747a470e0 con 0x559747be67a0 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f602dffb640 10 client.15123 handle_command_reply: tid=0 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 client.15123 shutdown 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 -- 172.21.15.103:0/1793388495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 msgr2=0x559747b7d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b7d180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f60340bc0c0 tx=0x7f60340ba640 comp rx=0 tx=0).stop 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 -- 172.21.15.103:0/1793388495 shutdown_connections 2024-09-05T02:02:07.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x559747be67a0 0x559747be54b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559747b67bb0 0x559747be0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559747b5d2f0 0x559747b7d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f602dffb640 0 client.15123 ms_handle_reset on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:07.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 --2- 172.21.15.103:0/1793388495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559747b5c9d0 0x559747b7d180 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 -- 172.21.15.103:0/1793388495 >> 172.21.15.103:0/1793388495 conn(0x559747b58f60 msgr2=0x559747b64710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:07.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 -- 172.21.15.103:0/1793388495 shutdown_connections 2024-09-05T02:02:07.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 1 -- 172.21.15.103:0/1793388495 wait complete. 2024-09-05T02:02:07.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.562+0000 7f604e8d9740 20 client.15123 trim_cache size 0 max 16384 2024-09-05T02:02:07.572 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24464, 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 71.365593051000005, 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 88, 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 136.60870915699999 2024-09-05T02:02:07.573 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:07.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 -- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60300a2c10 msgr2=0x7f60300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:07.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60300a2c10 0x7f60300a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f603c005fd0 tx=0x7f603c009820 comp rx=0 tx=0).stop 2024-09-05T02:02:07.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 -- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 msgr2=0x7f60441009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:07.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 0x7f60441009a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6034005940 tx=0x7f6034004270 comp rx=0 tx=0).stop 2024-09-05T02:02:07.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 -- 172.21.15.103:0/3586055960 shutdown_connections 2024-09-05T02:02:07.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f604410c170 0x7f6044101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60300a2c10 0x7f60300a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6044103660 0x7f6044100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 --2- 172.21.15.103:0/3586055960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6044102d40 0x7f60441009a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:07.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 -- 172.21.15.103:0/3586055960 >> 172.21.15.103:0/3586055960 conn(0x7f60440fc7a0 msgr2=0x7f60440fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:07.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 -- 172.21.15.103:0/3586055960 shutdown_connections 2024-09-05T02:02:07.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:07.563+0000 7f604c500640 1 -- 172.21.15.103:0/3586055960 wait complete. 2024-09-05T02:02:08.435 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T02:02:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:08 smithi167 ceph-mon[35122]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:08.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:08 smithi139 ceph-mon[34262]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:08 smithi103 ceph-mon[30902]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:08.682 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:09.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.132+0000 7fadb3870640 1 -- 172.21.15.103:0/1699615184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac102790 msgr2=0x7fadac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:09.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.132+0000 7fadb3870640 1 --2- 172.21.15.103:0/1699615184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac102790 0x7fadac1067e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009900 tx=0x7fad9c02a410 comp rx=0 tx=0).stop 2024-09-05T02:02:09.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.132+0000 7fadb3870640 1 -- 172.21.15.103:0/1699615184 shutdown_connections 2024-09-05T02:02:09.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.132+0000 7fadb3870640 1 --2- 172.21.15.103:0/1699615184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac106f10 0x7fadac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.132+0000 7fadb3870640 1 --2- 172.21.15.103:0/1699615184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac102790 0x7fadac1067e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.132+0000 7fadb3870640 1 --2- 172.21.15.103:0/1699615184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadac101de0 0x7fadac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.132+0000 7fadb3870640 1 -- 172.21.15.103:0/1699615184 >> 172.21.15.103:0/1699615184 conn(0x7fadac0fd8c0 msgr2=0x7fadac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:09.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.132+0000 7fadb3870640 1 -- 172.21.15.103:0/1699615184 shutdown_connections 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.132+0000 7fadb3870640 1 -- 172.21.15.103:0/1699615184 wait complete. 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb3870640 1 Processor -- start 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb3870640 1 -- start start 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb3870640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac101de0 0x7fadac198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb3870640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadac102790 0x7fadac198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb3870640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac106f10 0x7fadac19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb3870640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fadac111e40 con 0x7fadac101de0 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb3870640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fadac111cc0 con 0x7fadac102790 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb3870640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadac111fc0 con 0x7fadac106f10 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb160c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac101de0 0x7fadac198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb160c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac101de0 0x7fadac198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35016/0 (socket says 172.21.15.103:35016) 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb160c640 1 -- 172.21.15.103:0/2688535957 learned_addr learned my addr 172.21.15.103:0/2688535957 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac106f10 0x7fadac19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadac102790 0x7fadac198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb160c640 1 -- 172.21.15.103:0/2688535957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac106f10 msgr2=0x7fadac19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb160c640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac106f10 0x7fadac19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb160c640 1 -- 172.21.15.103:0/2688535957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadac102790 msgr2=0x7fadac198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb160c640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadac102790 0x7fadac198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb160c640 1 -- 172.21.15.103:0/2688535957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac19d3b0 con 0x7fadac101de0 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.133+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadac102790 0x7fadac198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.134+0000 7fadb160c640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac101de0 0x7fadac198400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fada800b4a0 tx=0x7fada800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.134+0000 7fada27fc640 1 -- 172.21.15.103:0/2688535957 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8014070 con 0x7fadac101de0 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.134+0000 7fadb3870640 1 -- 172.21.15.103:0/2688535957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac19d6a0 con 0x7fadac101de0 2024-09-05T02:02:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.134+0000 7fada27fc640 1 -- 172.21.15.103:0/2688535957 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada8010ac0 con 0x7fadac101de0 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.134+0000 7fada27fc640 1 -- 172.21.15.103:0/2688535957 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8017c30 con 0x7fadac101de0 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.134+0000 7fadb3870640 1 -- 172.21.15.103:0/2688535957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac1a4f80 con 0x7fadac101de0 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fada27fc640 1 -- 172.21.15.103:0/2688535957 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fada8017070 con 0x7fadac101de0 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb5c49740 1 Processor -- start 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb5c49740 1 -- start start 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d904262c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d904271a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d9042743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb5c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d9041c9090 con 0x55d904271fc0 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb5c49740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d904170a70 con 0x55d904262890 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb5c49740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d9041c5050 con 0x55d9042631b0 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb0e0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d9042743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb0e0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d9042743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35024/0 (socket says 172.21.15.103:35024) 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.137+0000 7fadb0e0b640 1 -- 172.21.15.103:0/4214024684 learned_addr learned my addr 172.21.15.103:0/4214024684 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d904262c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb160c640 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d904271a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fada27fc640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fad980a2c10 0x7fad980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb0e0b640 1 -- 172.21.15.103:0/4214024684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 msgr2=0x55d904271a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d904271a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb0e0b640 1 -- 172.21.15.103:0/4214024684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 msgr2=0x55d904262c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d904262c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb0e0b640 1 -- 172.21.15.103:0/4214024684 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90417d130 con 0x55d904271fc0 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fad980a2c10 0x7fad980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb160c640 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d904271a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fada27fc640 1 -- 172.21.15.103:0/2688535957 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fada80ceb00 con 0x7fadac101de0 2024-09-05T02:02:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d9042743b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fad9c02a930 tx=0x7fad9c031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d433524319f17a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fad980a2c10 0x7fad980a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fada4003a30 tx=0x7fada4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fad977fe640 1 -- 172.21.15.103:0/4214024684 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c03f070 con 0x55d904271fc0 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fad977fe640 1 -- 172.21.15.103:0/4214024684 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad9c0316a0 con 0x55d904271fc0 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.138+0000 7fad977fe640 1 -- 172.21.15.103:0/4214024684 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c035070 con 0x55d904271fc0 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 -- 172.21.15.103:0/4214024684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 msgr2=0x55d9042743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d9042743b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fad9c02a930 tx=0x7fad9c031e40 comp rx=0 tx=0).stop 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 -- 172.21.15.103:0/4214024684 shutdown_connections 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d9042743b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d904271a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 --2- 172.21.15.103:0/4214024684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d904262c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 -- 172.21.15.103:0/4214024684 >> 172.21.15.103:0/4214024684 conn(0x55d90425ef60 msgr2=0x55d90425f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 -- 172.21.15.103:0/4214024684 shutdown_connections 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 -- 172.21.15.103:0/4214024684 wait complete. 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.139+0000 7fadb5c49740 1 -- start start 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb5c49740 1 Processor -- start 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d9042717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d90426eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d90426f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb5c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d904274b30 con 0x55d904271fc0 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb5c49740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d904183f20 con 0x55d904262890 2024-09-05T02:02:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb5c49740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d904274cb0 con 0x55d9042631b0 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb5c49740 10 client.0 _ll_register_callbacks cb 0x55d90414d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb1e0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d90426f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb1e0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d90426f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35026/0 (socket says 172.21.15.103:35026) 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb160c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d9042717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb1e0d640 1 -- 172.21.15.103:0/1625846086 learned_addr learned my addr 172.21.15.103:0/1625846086 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.140+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d90426eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb1e0d640 1 -- 172.21.15.103:0/1625846086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 msgr2=0x55d90426eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d90426eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb1e0d640 1 -- 172.21.15.103:0/1625846086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 msgr2=0x55d9042717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d9042717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb1e0d640 1 -- 172.21.15.103:0/1625846086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d903f3a2b0 con 0x55d904271fc0 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb160c640 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d9042717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:02:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d90426f0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fada4004010 tx=0x7fada4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2363342ffe9827b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fad94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fad94ff9640 1 -- 172.21.15.103:0/1625846086 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4010b90 con 0x55d904271fc0 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fad94ff9640 1 -- 172.21.15.103:0/1625846086 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fada4010d30 con 0x55d904271fc0 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb5c49740 1 -- 172.21.15.103:0/1625846086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c0316a0 con 0x55d904271fc0 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fad94ff9640 1 -- 172.21.15.103:0/1625846086 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fada401f400 con 0x55d904271fc0 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb5c49740 10 client.15135 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fadb5c49740 1 -- 172.21.15.103:0/1625846086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d904280490 con 0x55d904271fc0 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.141+0000 7fad94ff9640 1 -- 172.21.15.103:0/1625846086 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad9c0316a0 con 0x55d904271fc0 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fad94ff9640 1 -- 172.21.15.103:0/1625846086 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fada401b720 con 0x55d904271fc0 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fadb5c49740 10 client.15135 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fadb5c49740 10 client.15135 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fadb5c49740 20 client.15135 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x55d904282b50 0x55d904283fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fadb5c49740 4 client.15135 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fadb5c49740 1 -- 172.21.15.103:0/1625846086 --> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9041d6600 con 0x55d904282b50 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fadb160c640 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x55d904282b50 0x55d904283fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fad94ff9640 10 client.15135 ms_handle_connect on v2:172.21.15.167:6838/1858062899 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.142+0000 7fadb160c640 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x55d904282b50 0x55d904283fc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d16dbbb5b0e3f41 server_cookie=9b23d4672e0ec719 in_seq=0 out_seq=0 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fad94ff9640 1 -- 172.21.15.103:0/1625846086 <== mds.1 v2:172.21.15.167:6838/1858062899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9041d6600 con 0x55d904282b50 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fad94ff9640 10 client.15135 handle_command_reply: tid=0 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 client.15135 shutdown 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 -- 172.21.15.103:0/1625846086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 msgr2=0x55d90426f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d90426f0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fada4004010 tx=0x7fada4004ab0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 -- 172.21.15.103:0/1625846086 shutdown_connections 2024-09-05T02:02:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x55d904282b50 0x55d904283fc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d90426f0a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d90426eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1625846086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d9042717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fad94ff9640 0 client.15135 ms_handle_reset on v2:172.21.15.167:6838/1858062899 2024-09-05T02:02:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 -- 172.21.15.103:0/1625846086 >> 172.21.15.103:0/1625846086 conn(0x55d90425ef60 msgr2=0x55d904264cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.143+0000 7fadb5c49740 1 -- 172.21.15.103:0/1625846086 shutdown_connections 2024-09-05T02:02:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.144+0000 7fadb5c49740 1 -- 172.21.15.103:0/1625846086 wait complete. 2024-09-05T02:02:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.144+0000 7fadb5c49740 20 client.15135 trim_cache size 0 max 16384 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.161+0000 7fadb5c49740 1 Processor -- start 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.161+0000 7fadb5c49740 1 -- start start 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.161+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d904282b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.161+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d9042838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.161+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d904282480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.161+0000 7fadb5c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d904282300 con 0x55d904271fc0 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.161+0000 7fadb5c49740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d9042829d0 con 0x55d904262890 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.161+0000 7fadb5c49740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d9042830a0 con 0x55d9042631b0 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb1e0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d904282480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb1e0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d904282480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35032/0 (socket says 172.21.15.103:35032) 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb1e0d640 1 -- 172.21.15.103:0/3219088026 learned_addr learned my addr 172.21.15.103:0/3219088026 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb160c640 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d904282b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d9042838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb1e0d640 1 -- 172.21.15.103:0/3219088026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 msgr2=0x55d9042838f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d9042838f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb1e0d640 1 -- 172.21.15.103:0/3219088026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 msgr2=0x55d904282b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d904282b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb1e0d640 1 -- 172.21.15.103:0/3219088026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d904274ca0 con 0x55d904271fc0 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d9042838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb160c640 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d904282b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d904282480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fada401bc10 tx=0x7fada40044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ba7d4a4207ca86 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fad96ffd640 1 -- 172.21.15.103:0/3219088026 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4037020 con 0x55d904271fc0 2024-09-05T02:02:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fad96ffd640 1 -- 172.21.15.103:0/3219088026 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada4024020 con 0x55d904271fc0 2024-09-05T02:02:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.162+0000 7fad96ffd640 1 -- 172.21.15.103:0/3219088026 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4023040 con 0x55d904271fc0 2024-09-05T02:02:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 -- 172.21.15.103:0/3219088026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 msgr2=0x55d904282480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d904282480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fada401bc10 tx=0x7fada40044f0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 -- 172.21.15.103:0/3219088026 shutdown_connections 2024-09-05T02:02:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904271fc0 0x55d904282480 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d9042838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 --2- 172.21.15.103:0/3219088026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904262890 0x55d904282b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 -- 172.21.15.103:0/3219088026 >> 172.21.15.103:0/3219088026 conn(0x55d90425ef60 msgr2=0x55d904259ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 -- 172.21.15.103:0/3219088026 shutdown_connections 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 -- 172.21.15.103:0/3219088026 wait complete. 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 -- start start 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.163+0000 7fadb5c49740 1 Processor -- start 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904262890 0x55d90428cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d9042851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb5c49740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904271fc0 0x55d904285720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb5c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d904283e40 con 0x55d904262890 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb5c49740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d904283770 con 0x55d904271fc0 2024-09-05T02:02:09.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb5c49740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d904284510 con 0x55d9042631b0 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb5c49740 10 client.0 _ll_register_callbacks cb 0x7fada4005bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb160c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904262890 0x55d90428cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb160c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904262890 0x55d90428cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35048/0 (socket says 172.21.15.103:35048) 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb160c640 1 -- 172.21.15.103:0/1654993165 learned_addr learned my addr 172.21.15.103:0/1654993165 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb160c640 1 -- 172.21.15.103:0/1654993165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 msgr2=0x55d9042851c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d9042851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb1e0d640 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904271fc0 0x55d904285720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb160c640 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d9042851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb160c640 1 -- 172.21.15.103:0/1654993165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904271fc0 msgr2=0x55d904285720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb160c640 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904271fc0 0x55d904285720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb160c640 1 -- 172.21.15.103:0/1654993165 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d904285f50 con 0x55d904262890 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fadb160c640 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904262890 0x55d90428cb00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fada809af40 tx=0x7fada8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad32b5b103decb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:09.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.164+0000 7fad94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fad94ff9640 1 -- 172.21.15.103:0/1654993165 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada801e040 con 0x55d904262890 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fad94ff9640 1 -- 172.21.15.103:0/1654993165 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fada80cc310 con 0x55d904262890 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fadb5c49740 1 -- 172.21.15.103:0/1654993165 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada4024020 con 0x55d904262890 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fad94ff9640 1 -- 172.21.15.103:0/1654993165 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fada809f360 con 0x55d904262890 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fadb5c49740 10 client.15141 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fadb5c49740 1 -- 172.21.15.103:0/1654993165 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9042ec2b0 con 0x55d904262890 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fad94ff9640 1 -- 172.21.15.103:0/1654993165 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada4024020 con 0x55d904262890 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fad94ff9640 1 -- 172.21.15.103:0/1654993165 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fada80a3790 con 0x55d904262890 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fadb5c49740 10 client.15141 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fadb5c49740 10 client.15141 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fadb5c49740 20 client.15141 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x55d9042f0e70 0x55d9042efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:09.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fadb5c49740 4 client.15141 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.165+0000 7fadb5c49740 1 -- 172.21.15.103:0/1654993165 --> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] -- command(tid 0: {"prefix": "status"}) -- 0x55d9041d0b10 con 0x55d9042f0e70 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x55d9042f0e70 0x55d9042efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb0e0b640 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x55d9042f0e70 0x55d9042efb80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c981caafaa41900 server_cookie=1cd70bca6134f7d in_seq=0 out_seq=0 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fad94ff9640 10 client.15141 ms_handle_connect on v2:172.21.15.167:6838/1858062899 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fad94ff9640 1 -- 172.21.15.103:0/1654993165 <== mds.1 v2:172.21.15.167:6838/1858062899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d9041d0b10 con 0x55d9042f0e70 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fad94ff9640 10 client.15141 handle_command_reply: tid=0 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb5c49740 1 client.15141 shutdown 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb5c49740 1 -- 172.21.15.103:0/1654993165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904262890 msgr2=0x55d90428cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904262890 0x55d90428cb00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fada809af40 tx=0x7fada8000bc0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb5c49740 1 -- 172.21.15.103:0/1654993165 shutdown_connections 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x55d9042f0e70 0x55d9042efb80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d904271fc0 0x55d904285720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9042631b0 0x55d9042851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fad94ff9640 0 client.15141 ms_handle_reset on v2:172.21.15.167:6838/1858062899 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb5c49740 1 --2- 172.21.15.103:0/1654993165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d904262890 0x55d90428cb00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.166+0000 7fadb5c49740 1 -- 172.21.15.103:0/1654993165 >> 172.21.15.103:0/1654993165 conn(0x55d90425ef60 msgr2=0x55d90425f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb5c49740 1 -- 172.21.15.103:0/1654993165 shutdown_connections 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb5c49740 1 -- 172.21.15.103:0/1654993165 wait complete. 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb5c49740 20 client.15141 trim_cache size 0 max 16384 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24466, 2024-09-05T02:02:09.176 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:09.177 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:09.177 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:09.177 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 73.974483462999999, 2024-09-05T02:02:09.177 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:09.177 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 88, 2024-09-05T02:02:09.177 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:09.177 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 135.60914126700001 2024-09-05T02:02:09.177 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:09.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 -- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fad980a2c10 msgr2=0x7fad980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fad980a2c10 0x7fad980a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fada4003a30 tx=0x7fada4004a40 comp rx=0 tx=0).stop 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 -- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac101de0 msgr2=0x7fadac198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac101de0 0x7fadac198400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fada800b4a0 tx=0x7fada800b960 comp rx=0 tx=0).stop 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 -- 172.21.15.103:0/2688535957 shutdown_connections 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fad980a2c10 0x7fad980a50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac106f10 0x7fadac19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadac102790 0x7fadac198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 --2- 172.21.15.103:0/2688535957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadac101de0 0x7fadac198400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 -- 172.21.15.103:0/2688535957 >> 172.21.15.103:0/2688535957 conn(0x7fadac0fd8c0 msgr2=0x7fadac0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 -- 172.21.15.103:0/2688535957 shutdown_connections 2024-09-05T02:02:09.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:09.167+0000 7fadb3870640 1 -- 172.21.15.103:0/2688535957 wait complete. 2024-09-05T02:02:09.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T02:02:10.138 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:10.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:10 smithi103 ceph-mon[30902]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:02:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:10 smithi167 ceph-mon[35122]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:02:10.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:10 smithi139 ceph-mon[34262]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:02:10.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 -- 172.21.15.103:0/152059255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c08103660 msgr2=0x7f2c0810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:10.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 --2- 172.21.15.103:0/152059255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c08103660 0x7f2c0810bb50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009900 tx=0x7f2bfc02a480 comp rx=0 tx=0).stop 2024-09-05T02:02:10.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 -- 172.21.15.103:0/152059255 shutdown_connections 2024-09-05T02:02:10.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 --2- 172.21.15.103:0/152059255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c0810c170 0x7f2c0810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 --2- 172.21.15.103:0/152059255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c08103660 0x7f2c0810bb50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 --2- 172.21.15.103:0/152059255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c08102d40 0x7f2c08103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 -- 172.21.15.103:0/152059255 >> 172.21.15.103:0/152059255 conn(0x7f2c080fc7a0 msgr2=0x7f2c080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:10.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 -- 172.21.15.103:0/152059255 shutdown_connections 2024-09-05T02:02:10.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 -- 172.21.15.103:0/152059255 wait complete. 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 Processor -- start 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.661+0000 7f2c0e891640 1 -- start start 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0e891640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c08102d40 0x7f2c08198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0e891640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c08103660 0x7f2c08198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0e891640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0810c170 0x7f2c0819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0e891640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08111e20 con 0x7f2c0810c170 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0e891640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08111ca0 con 0x7f2c08103660 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0e891640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08111fa0 con 0x7f2c08102d40 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0810c170 0x7f2c0819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0810c170 0x7f2c0819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60932/0 (socket says 172.21.15.103:60932) 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0ce2e640 1 -- 172.21.15.103:0/1837818823 learned_addr learned my addr 172.21.15.103:0/1837818823 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:10.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c07fff640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c08102d40 0x7f2c08198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c077fe640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c08103660 0x7f2c08198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0ce2e640 1 -- 172.21.15.103:0/1837818823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c08102d40 msgr2=0x7f2c08198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0ce2e640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c08102d40 0x7f2c08198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0ce2e640 1 -- 172.21.15.103:0/1837818823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c08103660 msgr2=0x7f2c08198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0ce2e640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c08103660 0x7f2c08198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0ce2e640 1 -- 172.21.15.103:0/1837818823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0819d2d0 con 0x7f2c0810c170 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0ce2e640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0810c170 0x7f2c0819cbf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000eaa0 tx=0x7f2c0000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c057fa640 1 -- 172.21.15.103:0/1837818823 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00004d20 con 0x7f2c0810c170 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c057fa640 1 -- 172.21.15.103:0/1837818823 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c00005020 con 0x7f2c0810c170 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0e891640 1 -- 172.21.15.103:0/1837818823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0819d5c0 con 0x7f2c0810c170 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c057fa640 1 -- 172.21.15.103:0/1837818823 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c000116b0 con 0x7f2c0810c170 2024-09-05T02:02:10.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.662+0000 7f2c0e891640 1 -- 172.21.15.103:0/1837818823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c081a4ea0 con 0x7f2c0810c170 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.665+0000 7f2c057fa640 1 -- 172.21.15.103:0/1837818823 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2c000051d0 con 0x7f2c0810c170 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.665+0000 7f2c10c6a740 1 Processor -- start 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.665+0000 7f2c10c6a740 1 -- start start 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.665+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fd7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.665+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fe2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.665+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 0x555911fe4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.665+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555911f3e090 con 0x555911fd79d0 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.665+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555911ee5a70 con 0x555911fd82f0 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.665+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555911f3a050 con 0x555911fe2bb0 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fd7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fd7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60942/0 (socket says 172.21.15.103:60942) 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c077fe640 1 -- 172.21.15.103:0/1313728203 learned_addr learned my addr 172.21.15.103:0/1313728203 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:10.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c07fff640 1 --2- 172.21.15.103:0/1313728203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fe2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c057fa640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2bf00a2c10 0x7f2bf00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c077fe640 1 -- 172.21.15.103:0/1313728203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 msgr2=0x555911fe4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c077fe640 1 --2- 172.21.15.103:0/1313728203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 0x555911fe4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c0ce2e640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2bf00a2c10 0x7f2bf00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c077fe640 1 -- 172.21.15.103:0/1313728203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 msgr2=0x555911fe2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c077fe640 1 --2- 172.21.15.103:0/1313728203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fe2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c077fe640 1 -- 172.21.15.103:0/1313728203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555911ef2130 con 0x555911fd79d0 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c057fa640 1 -- 172.21.15.103:0/1837818823 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f2c000cf700 con 0x7f2c0810c170 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c077fe640 1 --2- 172.21.15.103:0/1313728203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fd7db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc005ab0 tx=0x7f2bfc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9eb09e8e41d29e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2bee7fc640 1 -- 172.21.15.103:0/1313728203 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc035050 con 0x555911fd79d0 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2c0ce2e640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2bf00a2c10 0x7f2bf00a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2c0009b740 tx=0x7f2c00086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:10.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2bee7fc640 1 -- 172.21.15.103:0/1313728203 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bfc0316a0 con 0x555911fd79d0 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.666+0000 7f2bee7fc640 1 -- 172.21.15.103:0/1313728203 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0046c0 con 0x555911fd79d0 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.667+0000 7f2c10c6a740 1 -- 172.21.15.103:0/1313728203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 msgr2=0x555911fd7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.667+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/1313728203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fd7db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc005ab0 tx=0x7f2bfc005e50 comp rx=0 tx=0).stop 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.667+0000 7f2c10c6a740 1 -- 172.21.15.103:0/1313728203 shutdown_connections 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.667+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/1313728203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 0x555911fe4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.667+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/1313728203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fe2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.667+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/1313728203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fd7db0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.667+0000 7f2c10c6a740 1 -- 172.21.15.103:0/1313728203 >> 172.21.15.103:0/1313728203 conn(0x555911fd3f60 msgr2=0x555911fd4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.667+0000 7f2c10c6a740 1 -- 172.21.15.103:0/1313728203 shutdown_connections 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.667+0000 7f2c10c6a740 1 -- 172.21.15.103:0/1313728203 wait complete. 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c10c6a740 1 -- start start 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c10c6a740 1 Processor -- start 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fe2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fdf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 0x555911fdfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555911fe5780 con 0x555911fd79d0 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555911ef8f20 con 0x555911fd82f0 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555911fe5900 con 0x555911fe2bb0 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c10c6a740 10 client.0 _ll_register_callbacks cb 0x555911ec2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fe2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fe2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60950/0 (socket says 172.21.15.103:60950) 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c077fe640 1 -- 172.21.15.103:0/4217705394 learned_addr learned my addr 172.21.15.103:0/4217705394 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c07fff640 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fdf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c077fe640 1 -- 172.21.15.103:0/4217705394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 msgr2=0x555911fdfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c077fe640 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 0x555911fdfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c077fe640 1 -- 172.21.15.103:0/4217705394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 msgr2=0x555911fdf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:10.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c077fe640 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fdf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.668+0000 7f2c077fe640 1 -- 172.21.15.103:0/4217705394 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555911caf2b0 con 0x555911fd79d0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c077fe640 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fe2430 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc031440 tx=0x7f2bfc032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e0df94a76a4580 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2be77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2be77fe640 1 -- 172.21.15.103:0/4217705394 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc035040 con 0x555911fd79d0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2be77fe640 1 -- 172.21.15.103:0/4217705394 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f2bfc03d890 con 0x555911fd79d0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c10c6a740 1 -- 172.21.15.103:0/4217705394 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bfc0316a0 con 0x555911fd79d0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2be77fe640 1 -- 172.21.15.103:0/4217705394 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bfc057050 con 0x555911fd79d0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c10c6a740 10 client.15153 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c10c6a740 1 -- 172.21.15.103:0/4217705394 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555911ff10c0 con 0x555911fd79d0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2be77fe640 1 -- 172.21.15.103:0/4217705394 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bfc0316a0 con 0x555911fd79d0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2be77fe640 1 -- 172.21.15.103:0/4217705394 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f2bfc03abe0 con 0x555911fd79d0 2024-09-05T02:02:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c10c6a740 10 client.15153 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c10c6a740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c10c6a740 20 client.15153 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555911ff31c0 0x555911ff4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c10c6a740 4 client.15153 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.669+0000 7f2c10c6a740 1 -- 172.21.15.103:0/4217705394 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555911f4b600 con 0x555911ff31c0 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.670+0000 7f2c07fff640 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555911ff31c0 0x555911ff4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.670+0000 7f2c07fff640 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555911ff31c0 0x555911ff4630 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f01385bf63197040 server_cookie=98046f75b6540196 in_seq=0 out_seq=0 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.670+0000 7f2be77fe640 10 client.15153 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2be77fe640 1 -- 172.21.15.103:0/4217705394 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555911f4b600 con 0x555911ff31c0 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2be77fe640 10 client.15153 handle_command_reply: tid=0 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 client.15153 shutdown 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 -- 172.21.15.103:0/4217705394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 msgr2=0x555911fe2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fe2430 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc031440 tx=0x7f2bfc032f60 comp rx=0 tx=0).stop 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 -- 172.21.15.103:0/4217705394 shutdown_connections 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555911ff31c0 0x555911ff4630 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 0x555911fdfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fdf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/4217705394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911fe2430 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 -- 172.21.15.103:0/4217705394 >> 172.21.15.103:0/4217705394 conn(0x555911fd3f60 msgr2=0x555911fd9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2be77fe640 0 client.15153 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 -- 172.21.15.103:0/4217705394 shutdown_connections 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 1 -- 172.21.15.103:0/4217705394 wait complete. 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.671+0000 7f2c10c6a740 20 client.15153 trim_cache size 0 max 16384 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c10c6a740 1 Processor -- start 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c10c6a740 1 -- start start 2024-09-05T02:02:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911ff31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fde050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 0x555911ffc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555911ff2970 con 0x555911fd79d0 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555911ff3040 con 0x555911fd82f0 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555911ff3710 con 0x555911fe2bb0 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911ff31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911ff31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60966/0 (socket says 172.21.15.103:60966) 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c077fe640 1 -- 172.21.15.103:0/978197972 learned_addr learned my addr 172.21.15.103:0/978197972 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c077fe640 1 -- 172.21.15.103:0/978197972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 msgr2=0x555911ffc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c077fe640 1 --2- 172.21.15.103:0/978197972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 0x555911ffc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c077fe640 1 -- 172.21.15.103:0/978197972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 msgr2=0x555911fde050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c077fe640 1 --2- 172.21.15.103:0/978197972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fde050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c077fe640 1 -- 172.21.15.103:0/978197972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555911fd8120 con 0x555911fd79d0 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.689+0000 7f2c077fe640 1 --2- 172.21.15.103:0/978197972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911ff31c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc000c00 tx=0x7f2bfc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d380adb9fc21738 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2bedffb640 1 -- 172.21.15.103:0/978197972 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc035070 con 0x555911fd79d0 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2bedffb640 1 -- 172.21.15.103:0/978197972 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bfc0316a0 con 0x555911fd79d0 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2bedffb640 1 -- 172.21.15.103:0/978197972 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc03d8f0 con 0x555911fd79d0 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2c10c6a740 1 -- 172.21.15.103:0/978197972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 msgr2=0x555911ff31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/978197972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911ff31c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc000c00 tx=0x7f2bfc009800 comp rx=0 tx=0).stop 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2c10c6a740 1 -- 172.21.15.103:0/978197972 shutdown_connections 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/978197972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fe2bb0 0x555911ffc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/978197972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fd82f0 0x555911fde050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/978197972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd79d0 0x555911ff31c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2c10c6a740 1 -- 172.21.15.103:0/978197972 >> 172.21.15.103:0/978197972 conn(0x555911fd3f60 msgr2=0x555911fcece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2c10c6a740 1 -- 172.21.15.103:0/978197972 shutdown_connections 2024-09-05T02:02:10.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.690+0000 7f2c10c6a740 1 -- 172.21.15.103:0/978197972 wait complete. 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c10c6a740 1 -- start start 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c10c6a740 1 Processor -- start 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fd79d0 0x555911ff5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd82f0 0x555911ff62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c10c6a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fe2bb0 0x55591205b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555911ff44b0 con 0x555911fd82f0 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555911ff3de0 con 0x555911fe2bb0 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c10c6a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555911ff4b80 con 0x555911fd79d0 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c10c6a740 10 client.0 _ll_register_callbacks cb 0x7f2bf800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd82f0 0x555911ff62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd82f0 0x555911ff62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60972/0 (socket says 172.21.15.103:60972) 2024-09-05T02:02:10.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c07fff640 1 -- 172.21.15.103:0/579726436 learned_addr learned my addr 172.21.15.103:0/579726436 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c077fe640 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fe2bb0 0x55591205b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c077fe640 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fd79d0 0x555911ff5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c07fff640 1 -- 172.21.15.103:0/579726436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fd79d0 msgr2=0x555911ff5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c07fff640 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fd79d0 0x555911ff5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c07fff640 1 -- 172.21.15.103:0/579726436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fe2bb0 msgr2=0x55591205b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c07fff640 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fe2bb0 0x55591205b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.691+0000 7f2c07fff640 1 -- 172.21.15.103:0/579726436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55591205be80 con 0x555911fd82f0 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c07fff640 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd82f0 0x555911ff62d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x555911fe0dc0 tx=0x7f2bf8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5259646b8c3b64a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2be77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2be77fe640 1 -- 172.21.15.103:0/579726436 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8005460 con 0x555911fd82f0 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2be77fe640 1 -- 172.21.15.103:0/579726436 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f2bf8005600 con 0x555911fd82f0 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c10c6a740 1 -- 172.21.15.103:0/579726436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bfc0316a0 con 0x555911fd82f0 2024-09-05T02:02:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2be77fe640 1 -- 172.21.15.103:0/579726436 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bf80047a0 con 0x555911fd82f0 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c10c6a740 10 client.15159 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c10c6a740 1 -- 172.21.15.103:0/579726436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55591205d210 con 0x555911fd82f0 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24497, 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:10.715 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 115.325963243, 2024-09-05T02:02:10.716 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:10.716 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 88, 2024-09-05T02:02:10.716 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:10.716 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 127.01157009000001 2024-09-05T02:02:10.716 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2be77fe640 1 -- 172.21.15.103:0/579726436 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bfc0316a0 con 0x555911fd82f0 2024-09-05T02:02:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2be77fe640 1 -- 172.21.15.103:0/579726436 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f2bf801eb30 con 0x555911fd82f0 2024-09-05T02:02:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c10c6a740 10 client.15159 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c10c6a740 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T02:02:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c10c6a740 20 client.15159 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555912061e20 0x555912060b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c10c6a740 4 client.15159 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.692+0000 7f2c10c6a740 1 -- 172.21.15.103:0/579726436 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "status"}) -- 0x555911f45b10 con 0x555912061e20 2024-09-05T02:02:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c077fe640 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555912061e20 0x555912060b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c077fe640 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555912061e20 0x555912060b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7580d09ee1ff6bbc server_cookie=b2f5d0ef84fe076 in_seq=0 out_seq=0 2024-09-05T02:02:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2be77fe640 10 client.15159 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2be77fe640 1 -- 172.21.15.103:0/579726436 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555911f45b10 con 0x555912061e20 2024-09-05T02:02:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2be77fe640 10 client.15159 handle_command_reply: tid=0 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c10c6a740 1 client.15159 shutdown 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c10c6a740 1 -- 172.21.15.103:0/579726436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd82f0 msgr2=0x555911ff62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd82f0 0x555911ff62d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x555911fe0dc0 tx=0x7f2bf8010800 comp rx=0 tx=0).stop 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c10c6a740 1 -- 172.21.15.103:0/579726436 shutdown_connections 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555912061e20 0x555912060b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555911fe2bb0 0x55591205b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555911fd82f0 0x555911ff62d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2be77fe640 0 client.15159 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c10c6a740 1 --2- 172.21.15.103:0/579726436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555911fd79d0 0x555911ff5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.693+0000 7f2c10c6a740 1 -- 172.21.15.103:0/579726436 >> 172.21.15.103:0/579726436 conn(0x555911fd3f60 msgr2=0x555911fddc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c10c6a740 1 -- 172.21.15.103:0/579726436 shutdown_connections 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c10c6a740 1 -- 172.21.15.103:0/579726436 wait complete. 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c10c6a740 20 client.15159 trim_cache size 0 max 16384 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 -- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2bf00a2c10 msgr2=0x7f2bf00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2bf00a2c10 0x7f2bf00a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2c0009b740 tx=0x7f2c00086040 comp rx=0 tx=0).stop 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 -- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0810c170 msgr2=0x7f2c0819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0810c170 0x7f2c0819cbf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000eaa0 tx=0x7f2c0000eea0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 -- 172.21.15.103:0/1837818823 shutdown_connections 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0810c170 0x7f2c0819cbf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c08103660 0x7f2c08198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2bf00a2c10 0x7f2bf00a50d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 --2- 172.21.15.103:0/1837818823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c08102d40 0x7f2c08198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 -- 172.21.15.103:0/1837818823 >> 172.21.15.103:0/1837818823 conn(0x7f2c080fc7a0 msgr2=0x7f2c080fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:10.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 -- 172.21.15.103:0/1837818823 shutdown_connections 2024-09-05T02:02:10.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:10.694+0000 7f2c0e891640 1 -- 172.21.15.103:0/1837818823 wait complete. 2024-09-05T02:02:11.513 INFO:tasks.cephadm:Setup complete, yielding 2024-09-05T02:02:11.513 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T02:02:11.523 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-05T02:02:11.524 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch status' 2024-09-05T02:02:11.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:11 smithi167 ceph-mon[35122]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:02:11.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:11 smithi139 ceph-mon[34262]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:02:11.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:11 smithi103 ceph-mon[30902]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:02:11.785 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:12 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:02:12.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:12 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:02:12.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:12 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:02:13.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.018+0000 7f002030b640 1 -- 172.21.15.103:0/1189154997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 msgr2=0x7f0018100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:13.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.018+0000 7f002030b640 1 --2- 172.21.15.103:0/1189154997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 0x7f0018100f10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0014009900 tx=0x7f001402a3f0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.018+0000 7f002030b640 1 -- 172.21.15.103:0/1189154997 shutdown_connections 2024-09-05T02:02:13.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.018+0000 7f002030b640 1 --2- 172.21.15.103:0/1189154997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00181026d0 0x7f0018108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.018+0000 7f002030b640 1 --2- 172.21.15.103:0/1189154997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0018101d10 0x7f0018102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.018+0000 7f002030b640 1 --2- 172.21.15.103:0/1189154997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 0x7f0018100f10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.018+0000 7f002030b640 1 -- 172.21.15.103:0/1189154997 >> 172.21.15.103:0/1189154997 conn(0x7f00180fc2a0 msgr2=0x7f00180fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.018+0000 7f002030b640 1 -- 172.21.15.103:0/1189154997 shutdown_connections 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.018+0000 7f002030b640 1 -- 172.21.15.103:0/1189154997 wait complete. 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f002030b640 1 Processor -- start 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f002030b640 1 -- start start 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f002030b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 0x7f0018197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f002030b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0018101d10 0x7f00181983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f002030b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00181026d0 0x7f001819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f002030b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f001810d730 con 0x7f0018100b10 2024-09-05T02:02:13.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f002030b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f001810d5b0 con 0x7f00181026d0 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f002030b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f001810d8b0 con 0x7f0018101d10 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001e0a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 0x7f0018197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001e0a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 0x7f0018197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60990/0 (socket says 172.21.15.103:60990) 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001e0a7640 1 -- 172.21.15.103:0/149345175 learned_addr learned my addr 172.21.15.103:0/149345175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001e8a8640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00181026d0 0x7f001819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001d8a6640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0018101d10 0x7f00181983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001e0a7640 1 -- 172.21.15.103:0/149345175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0018101d10 msgr2=0x7f00181983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001e0a7640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0018101d10 0x7f00181983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001e0a7640 1 -- 172.21.15.103:0/149345175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00181026d0 msgr2=0x7f001819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001e0a7640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00181026d0 0x7f001819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.019+0000 7f001e0a7640 1 -- 172.21.15.103:0/149345175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001819faa0 con 0x7f0018100b10 2024-09-05T02:02:13.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.020+0000 7f001e0a7640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 0x7f0018197e70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f00140314e0 tx=0x7f0014031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.020+0000 7f000f7fe640 1 -- 172.21.15.103:0/149345175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001403f030 con 0x7f0018100b10 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.020+0000 7f000f7fe640 1 -- 172.21.15.103:0/149345175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00140044b0 con 0x7f0018100b10 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.020+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001819fd30 con 0x7f0018100b10 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.020+0000 7f000f7fe640 1 -- 172.21.15.103:0/149345175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014035070 con 0x7f0018100b10 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.020+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00181a01c0 con 0x7f0018100b10 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.021+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0018107ff0 con 0x7f0018100b10 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.022+0000 7f000f7fe640 1 -- 172.21.15.103:0/149345175 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0014042030 con 0x7f0018100b10 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.022+0000 7f000f7fe640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f00040a2c40 0x7f00040a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.023+0000 7f001d8a6640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f00040a2c40 0x7f00040a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.023+0000 7f000f7fe640 1 -- 172.21.15.103:0/149345175 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f00140ef9f0 con 0x7f0018100b10 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.023+0000 7f001d8a6640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f00040a2c40 0x7f00040a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0018199390 tx=0x7f0008005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:13.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.027+0000 7f000f7fe640 1 -- 172.21.15.103:0/149345175 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00140bbe40 con 0x7f0018100b10 2024-09-05T02:02:13.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.241+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0018062630 con 0x7f00040a2c40 2024-09-05T02:02:13.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.242+0000 7f000f7fe640 1 -- 172.21.15.103:0/149345175 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0018062630 con 0x7f00040a2c40 2024-09-05T02:02:13.244 INFO:teuthology.orchestra.run.smithi103.stdout:Backend: cephadm 2024-09-05T02:02:13.244 INFO:teuthology.orchestra.run.smithi103.stdout:Available: Yes 2024-09-05T02:02:13.244 INFO:teuthology.orchestra.run.smithi103.stdout:Paused: No 2024-09-05T02:02:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f00040a2c40 msgr2=0x7f00040a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f00040a2c40 0x7f00040a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0018199390 tx=0x7f0008005f90 comp rx=0 tx=0).stop 2024-09-05T02:02:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 msgr2=0x7f0018197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 0x7f0018197e70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f00140314e0 tx=0x7f0014031a30 comp rx=0 tx=0).stop 2024-09-05T02:02:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 shutdown_connections 2024-09-05T02:02:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00181026d0 0x7f001819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f00040a2c40 0x7f00040a5100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0018101d10 0x7f00181983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 --2- 172.21.15.103:0/149345175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0018100b10 0x7f0018197e70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.244+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 >> 172.21.15.103:0/149345175 conn(0x7f00180fc2a0 msgr2=0x7f00181051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.245+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 shutdown_connections 2024-09-05T02:02:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:13.245+0000 7f002030b640 1 -- 172.21.15.103:0/149345175 wait complete. 2024-09-05T02:02:13.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:13.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:13.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:13.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:13.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:13 smithi103 ceph-mon[30902]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:13.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:13 smithi103 ceph-mon[30902]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:13.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:13.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:13 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:13.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-05T02:02:14.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:13 smithi167 ceph-mon[35122]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:14.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:13 smithi167 ceph-mon[35122]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:14.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:13 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:13 smithi139 ceph-mon[34262]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:14.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:13 smithi139 ceph-mon[34262]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:14.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:13 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:14.191 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.680+0000 7f02444f2640 1 -- 172.21.15.103:0/3447352556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 msgr2=0x7f023c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.680+0000 7f02444f2640 1 --2- 172.21.15.103:0/3447352556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 0x7f023c102190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f022c009900 tx=0x7f022c02a410 comp rx=0 tx=0).stop 2024-09-05T02:02:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 -- 172.21.15.103:0/3447352556 shutdown_connections 2024-09-05T02:02:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 --2- 172.21.15.103:0/3447352556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023c1026d0 0x7f023c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 --2- 172.21.15.103:0/3447352556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 0x7f023c102190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 --2- 172.21.15.103:0/3447352556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f023c100b10 0x7f023c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 -- 172.21.15.103:0/3447352556 >> 172.21.15.103:0/3447352556 conn(0x7f023c0fc2a0 msgr2=0x7f023c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 -- 172.21.15.103:0/3447352556 shutdown_connections 2024-09-05T02:02:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 -- 172.21.15.103:0/3447352556 wait complete. 2024-09-05T02:02:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 Processor -- start 2024-09-05T02:02:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 -- start start 2024-09-05T02:02:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023c100b10 0x7f023c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 0x7f023c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.681+0000 7f02444f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f023c1026d0 0x7f023c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f02444f2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f023c10c640 con 0x7f023c101d10 2024-09-05T02:02:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f02444f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f023c10c4c0 con 0x7f023c1026d0 2024-09-05T02:02:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f02444f2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f023c10c7c0 con 0x7f023c100b10 2024-09-05T02:02:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0241a8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 0x7f023c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0241a8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 0x7f023c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32780/0 (socket says 172.21.15.103:32780) 2024-09-05T02:02:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0241a8d640 1 -- 172.21.15.103:0/1136953782 learned_addr learned my addr 172.21.15.103:0/1136953782 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0242a8f640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f023c1026d0 0x7f023c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f024228e640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023c100b10 0x7f023c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0241a8d640 1 -- 172.21.15.103:0/1136953782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023c100b10 msgr2=0x7f023c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0241a8d640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023c100b10 0x7f023c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0241a8d640 1 -- 172.21.15.103:0/1136953782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f023c1026d0 msgr2=0x7f023c19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0241a8d640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f023c1026d0 0x7f023c19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0241a8d640 1 -- 172.21.15.103:0/1136953782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c19fa80 con 0x7f023c101d10 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f0241a8d640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 0x7f023c19cb50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f022c004770 tx=0x7f022c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f02337fe640 1 -- 172.21.15.103:0/1136953782 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c03f030 con 0x7f023c101d10 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f02337fe640 1 -- 172.21.15.103:0/1136953782 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f022c003cb0 con 0x7f023c101d10 2024-09-05T02:02:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023c19fcb0 con 0x7f023c101d10 2024-09-05T02:02:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f02337fe640 1 -- 172.21.15.103:0/1136953782 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c035070 con 0x7f023c101d10 2024-09-05T02:02:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.682+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023c1a0190 con 0x7f023c101d10 2024-09-05T02:02:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.683+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023c107f50 con 0x7f023c101d10 2024-09-05T02:02:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.685+0000 7f02337fe640 1 -- 172.21.15.103:0/1136953782 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f022c042030 con 0x7f023c101d10 2024-09-05T02:02:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.685+0000 7f02337fe640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f02280a2c90 0x7f02280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.685+0000 7f024228e640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f02280a2c90 0x7f02280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.685+0000 7f02337fe640 1 -- 172.21.15.103:0/1136953782 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f022c0eff70 con 0x7f023c101d10 2024-09-05T02:02:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.686+0000 7f024228e640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f02280a2c90 0x7f02280a5150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f023c101b70 tx=0x7f023800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.688+0000 7f02337fe640 1 -- 172.21.15.103:0/1136953782 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f022c0bc3c0 con 0x7f023c101d10 2024-09-05T02:02:14.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.864+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f023c062630 con 0x7f02280a2c90 2024-09-05T02:02:14.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.877+0000 7f02337fe640 1 -- 172.21.15.103:0/1136953782 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f023c062630 con 0x7f02280a2c90 2024-09-05T02:02:14.878 INFO:teuthology.orchestra.run.smithi103.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-05T02:02:14.878 INFO:teuthology.orchestra.run.smithi103.stdout:mds.a smithi103 running (2m) 1s ago 2m 18.1M - 19.1.1-259-g91c8a7e5 77770cf7e895 9a2f9e3e93e8 2024-09-05T02:02:14.878 INFO:teuthology.orchestra.run.smithi103.stdout:mds.b smithi139 running (2m) 2s ago 2m 29.3M - 19.1.1-259-g91c8a7e5 77770cf7e895 458b198b21b4 2024-09-05T02:02:14.878 INFO:teuthology.orchestra.run.smithi103.stdout:mds.c smithi167 running (2m) 1s ago 2m 21.7M - 19.1.1-259-g91c8a7e5 77770cf7e895 61d614251096 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mds.d smithi103 running (2m) 1s ago 2m 14.6M - 19.1.1-259-g91c8a7e5 77770cf7e895 eae91d00bc66 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mds.e smithi139 running (2m) 2s ago 2m 19.5M - 19.1.1-259-g91c8a7e5 77770cf7e895 e04809ef0a72 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mds.f smithi167 running (2m) 1s ago 2m 19.5M - 19.1.1-259-g91c8a7e5 77770cf7e895 044031dfc5a4 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mds.g smithi103 running (2m) 1s ago 2m 18.4M - 19.1.1-259-g91c8a7e5 77770cf7e895 e264f70f4e6e 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mds.h smithi139 running (2m) 2s ago 2m 18.4M - 19.1.1-259-g91c8a7e5 77770cf7e895 1fb4b4b48fb4 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mds.i smithi167 running (2m) 1s ago 2m 21.2M - 19.1.1-259-g91c8a7e5 77770cf7e895 6155c037fa44 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mds.j smithi103 running (2m) 1s ago 2m 14.8M - 19.1.1-259-g91c8a7e5 77770cf7e895 b371690aa115 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mds.k smithi139 running (2m) 2s ago 2m 18.5M - 19.1.1-259-g91c8a7e5 77770cf7e895 345bdedd6485 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mds.l smithi167 running (2m) 1s ago 2m 18.4M - 19.1.1-259-g91c8a7e5 77770cf7e895 b511a5265352 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.x smithi103 *:9283,8765 running (9m) 1s ago 9m 509M - 19.1.1-259-g91c8a7e5 77770cf7e895 a70b6e132a43 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.y smithi139 *:8443,8765 running (7m) 2s ago 7m 443M - 19.1.1-259-g91c8a7e5 77770cf7e895 a84e3496d589 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.z smithi167 *:8443,8765 running (7m) 1s ago 7m 442M - 19.1.1-259-g91c8a7e5 77770cf7e895 88fae74ca532 2024-09-05T02:02:14.879 INFO:teuthology.orchestra.run.smithi103.stdout:mon.a smithi103 running (9m) 1s ago 9m 52.6M 2048M 19.1.1-259-g91c8a7e5 77770cf7e895 462eb3fe0099 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:mon.b smithi139 running (8m) 2s ago 8m 41.9M 2048M 19.1.1-259-g91c8a7e5 77770cf7e895 56ef77362ff4 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:mon.c smithi167 running (8m) 1s ago 8m 42.5M 2048M 19.1.1-259-g91c8a7e5 77770cf7e895 f4b9c06b2ea3 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 smithi103 running (7m) 1s ago 7m 57.3M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 d9662f06b9d2 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 smithi139 running (7m) 2s ago 7m 55.5M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 6249536e978d 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 smithi167 running (6m) 1s ago 6m 46.5M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 9611305f997b 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 smithi103 running (6m) 1s ago 6m 53.2M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 507c8e11250a 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 smithi139 running (5m) 2s ago 5m 52.6M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 129e6d6e46df 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 smithi167 running (5m) 1s ago 5m 54.6M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 0b46b5de9083 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 smithi103 running (5m) 1s ago 5m 44.4M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 dc9e50e94de0 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 smithi139 running (4m) 2s ago 4m 45.6M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 29773379962c 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 smithi167 running (4m) 1s ago 4m 44.3M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 67fb60d52004 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 smithi103 running (3m) 1s ago 3m 51.8M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 f4dd94a8e948 2024-09-05T02:02:14.880 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 smithi139 running (3m) 2s ago 3m 43.2M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 f21d33f61435 2024-09-05T02:02:14.881 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 smithi167 running (3m) 1s ago 3m 50.8M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 198c1c2a4a35 2024-09-05T02:02:14.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f02280a2c90 msgr2=0x7f02280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:14.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f02280a2c90 0x7f02280a5150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f023c101b70 tx=0x7f023800b390 comp rx=0 tx=0).stop 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 msgr2=0x7f023c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 0x7f023c19cb50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f022c004770 tx=0x7f022c0042f0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 shutdown_connections 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f023c1026d0 0x7f023c19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f023c101d10 0x7f023c19cb50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f02280a2c90 0x7f02280a5150 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 --2- 172.21.15.103:0/1136953782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023c100b10 0x7f023c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 >> 172.21.15.103:0/1136953782 conn(0x7f023c0fc2a0 msgr2=0x7f023c105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 shutdown_connections 2024-09-05T02:02:14.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:14.880+0000 7f02444f2640 1 -- 172.21.15.103:0/1136953782 wait complete. 2024-09-05T02:02:15.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:02:15.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:02:15.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:15.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:02:15.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:02:15.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:15.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:02:15.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:02:15.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:02:15.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-05T02:02:15.874 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:15.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:15 smithi103 ceph-mon[30902]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:15.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:15 smithi103 ceph-mon[30902]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:02:16.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:15 smithi139 ceph-mon[34262]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:16.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:15 smithi139 ceph-mon[34262]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:02:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.346+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/4085378587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d41078c0 msgr2=0x7fe1d4109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.346+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/4085378587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d41078c0 0x7fe1d4109d10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009ca0 tx=0x7fe1cc02a8c0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.346+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/4085378587 shutdown_connections 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.346+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/4085378587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d41078c0 0x7fe1d4109d10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.346+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/4085378587 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1d4104f90 0x7fe1d4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.346+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/4085378587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1d4102660 0x7fe1d4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.346+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/4085378587 >> 172.21.15.103:0/4085378587 conn(0x7fe1d40fc280 msgr2=0x7fe1d40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.346+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/4085378587 shutdown_connections 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.346+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/4085378587 wait complete. 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d8ee4640 1 Processor -- start 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d8ee4640 1 -- start start 2024-09-05T02:02:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d8ee4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d4102660 0x7fe1d4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d8ee4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1d4104f90 0x7fe1d4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d8ee4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1d41078c0 0x7fe1d419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d8ee4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d410d730 con 0x7fe1d4102660 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d8ee4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d410d5b0 con 0x7fe1d4104f90 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d8ee4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d410d8b0 con 0x7fe1d41078c0 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d4102660 0x7fe1d4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d4102660 0x7fe1d4197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32800/0 (socket says 172.21.15.103:32800) 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d259c640 1 -- 172.21.15.103:0/1066017972 learned_addr learned my addr 172.21.15.103:0/1066017972 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d1d9b640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1d4104f90 0x7fe1d4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d2d9d640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1d41078c0 0x7fe1d419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d259c640 1 -- 172.21.15.103:0/1066017972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1d41078c0 msgr2=0x7fe1d419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d259c640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1d41078c0 0x7fe1d419f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d259c640 1 -- 172.21.15.103:0/1066017972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1d4104f90 msgr2=0x7fe1d4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d259c640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1d4104f90 0x7fe1d4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.347+0000 7fe1d259c640 1 -- 172.21.15.103:0/1066017972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d419fa40 con 0x7fe1d4102660 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.348+0000 7fe1d259c640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d4102660 0x7fe1d4197e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400e9f0 tx=0x7fe1c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.348+0000 7fe1c37fe640 1 -- 172.21.15.103:0/1066017972 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c400cd80 con 0x7fe1d4102660 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.348+0000 7fe1c37fe640 1 -- 172.21.15.103:0/1066017972 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1c40054d0 con 0x7fe1d4102660 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.348+0000 7fe1c37fe640 1 -- 172.21.15.103:0/1066017972 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4010490 con 0x7fe1d4102660 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.348+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d419fd30 con 0x7fe1d4102660 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.348+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d41a01c0 con 0x7fe1d4102660 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.349+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d40fdd70 con 0x7fe1d4102660 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.350+0000 7fe1c37fe640 1 -- 172.21.15.103:0/1066017972 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe1c4005020 con 0x7fe1d4102660 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.351+0000 7fe1c37fe640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe1bc0a2cc0 0x7fe1bc0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:16.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.351+0000 7fe1c37fe640 1 -- 172.21.15.103:0/1066017972 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fe1c40cf8a0 con 0x7fe1d4102660 2024-09-05T02:02:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.351+0000 7fe1d1d9b640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe1bc0a2cc0 0x7fe1bc0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.351+0000 7fe1d1d9b640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe1bc0a2cc0 0x7fe1bc0a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4199330 tx=0x7fe1c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.355+0000 7fe1c37fe640 1 -- 172.21.15.103:0/1066017972 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1c409c340 con 0x7fe1d4102660 2024-09-05T02:02:16.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:15 smithi167 ceph-mon[35122]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:16.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:15 smithi167 ceph-mon[35122]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:02:16.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.529+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe1d4062630 con 0x7fe1bc0a2cc0 2024-09-05T02:02:16.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.535+0000 7fe1c37fe640 1 -- 172.21.15.103:0/1066017972 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe1d4062630 con 0x7fe1bc0a2cc0 2024-09-05T02:02:16.535 INFO:teuthology.orchestra.run.smithi103.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-05T02:02:16.536 INFO:teuthology.orchestra.run.smithi103.stdout:mds.all 12/12 3s ago 2m smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-09-05T02:02:16.536 INFO:teuthology.orchestra.run.smithi103.stdout:mgr 3/3 3s ago 8m smithi103=x;smithi139=y;smithi167=z;count:3 2024-09-05T02:02:16.536 INFO:teuthology.orchestra.run.smithi103.stdout:mon 3/3 3s ago 8m smithi103:172.21.15.103=a;smithi139:172.21.15.139=b;smithi167:172.21.15.167=c;count:3 2024-09-05T02:02:16.536 INFO:teuthology.orchestra.run.smithi103.stdout:osd 12 3s ago - 2024-09-05T02:02:16.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe1bc0a2cc0 msgr2=0x7fe1bc0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe1bc0a2cc0 0x7fe1bc0a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4199330 tx=0x7fe1c8005f90 comp rx=0 tx=0).stop 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d4102660 msgr2=0x7fe1d4197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d4102660 0x7fe1d4197e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400e9f0 tx=0x7fe1c400edf0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 shutdown_connections 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe1bc0a2cc0 0x7fe1bc0a5180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1d41078c0 0x7fe1d419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1d4104f90 0x7fe1d4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 --2- 172.21.15.103:0/1066017972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1d4102660 0x7fe1d4197e10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 >> 172.21.15.103:0/1066017972 conn(0x7fe1d40fc280 msgr2=0x7fe1d4102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 shutdown_connections 2024-09-05T02:02:16.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:16.537+0000 7fe1d8ee4640 1 -- 172.21.15.103:0/1066017972 wait complete. 2024-09-05T02:02:17.280 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-05T02:02:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.023+0000 7f703ac50640 1 -- 172.21.15.103:0/3614225191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 msgr2=0x7f7034100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.023+0000 7f703ac50640 1 --2- 172.21.15.103:0/3614225191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 0x7f7034100f10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7028009900 tx=0x7f702802a410 comp rx=0 tx=0).stop 2024-09-05T02:02:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.023+0000 7f703ac50640 1 -- 172.21.15.103:0/3614225191 shutdown_connections 2024-09-05T02:02:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.023+0000 7f703ac50640 1 --2- 172.21.15.103:0/3614225191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70341026d0 0x7f7034108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.023+0000 7f703ac50640 1 --2- 172.21.15.103:0/3614225191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7034101d10 0x7f7034102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.023+0000 7f703ac50640 1 --2- 172.21.15.103:0/3614225191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 0x7f7034100f10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.023+0000 7f703ac50640 1 -- 172.21.15.103:0/3614225191 >> 172.21.15.103:0/3614225191 conn(0x7f70340fc2a0 msgr2=0x7f70340fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.023+0000 7f703ac50640 1 -- 172.21.15.103:0/3614225191 shutdown_connections 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.023+0000 7f703ac50640 1 -- 172.21.15.103:0/3614225191 wait complete. 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f703ac50640 1 Processor -- start 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f703ac50640 1 -- start start 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f703ac50640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 0x7f7034197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f703ac50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7034101d10 0x7f70341983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f703ac50640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70341026d0 0x7f703419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f703ac50640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f703410c670 con 0x7f7034100b10 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f703ac50640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f703410c4f0 con 0x7f7034101d10 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f703ac50640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f703410c7f0 con 0x7f70341026d0 2024-09-05T02:02:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70389ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 0x7f7034197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70389ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 0x7f7034197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32820/0 (socket says 172.21.15.103:32820) 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70389ec640 1 -- 172.21.15.103:0/1806625071 learned_addr learned my addr 172.21.15.103:0/1806625071 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f7033fff640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7034101d10 0x7f70341983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70391ed640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70341026d0 0x7f703419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70389ec640 1 -- 172.21.15.103:0/1806625071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70341026d0 msgr2=0x7f703419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70389ec640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70341026d0 0x7f703419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70389ec640 1 -- 172.21.15.103:0/1806625071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7034101d10 msgr2=0x7f70341983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70389ec640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7034101d10 0x7f70341983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70389ec640 1 -- 172.21.15.103:0/1806625071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703419fae0 con 0x7f7034100b10 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70391ed640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70341026d0 0x7f703419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.024+0000 7f70389ec640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 0x7f7034197eb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f702802a930 tx=0x7f7028003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.025+0000 7f7031ffb640 1 -- 172.21.15.103:0/1806625071 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702803f070 con 0x7f7034100b10 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.025+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703419fd70 con 0x7f7034100b10 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.025+0000 7f7031ffb640 1 -- 172.21.15.103:0/1806625071 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7028031e40 con 0x7f7034100b10 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.025+0000 7f7031ffb640 1 -- 172.21.15.103:0/1806625071 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028035030 con 0x7f7034100b10 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.025+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70341a01e0 con 0x7f7034100b10 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.026+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70341a04d0 con 0x7f7034100b10 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.027+0000 7f7031ffb640 1 -- 172.21.15.103:0/1806625071 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7028042070 con 0x7f7034100b10 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.027+0000 7f7031ffb640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240a2c40 0x7f70240a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.028+0000 7f7033fff640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240a2c40 0x7f70240a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.028+0000 7f7031ffb640 1 -- 172.21.15.103:0/1806625071 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f70280efee0 con 0x7f7034100b10 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.028+0000 7f7033fff640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240a2c40 0x7f70240a5100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70341993d0 tx=0x7f702000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.032+0000 7f7031ffb640 1 -- 172.21.15.103:0/1806625071 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70280bc980 con 0x7f7034100b10 2024-09-05T02:02:18.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.206+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f703406af30 con 0x7f70240a2c40 2024-09-05T02:02:18.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:18 smithi103 ceph-mon[30902]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:18.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:18 smithi103 ceph-mon[30902]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:18.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.207+0000 7f7031ffb640 1 -- 172.21.15.103:0/1806625071 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f703406af30 con 0x7f70240a2c40 2024-09-05T02:02:18.208 INFO:teuthology.orchestra.run.smithi103.stdout:HOST ADDR LABELS STATUS 2024-09-05T02:02:18.208 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 172.21.15.103 2024-09-05T02:02:18.208 INFO:teuthology.orchestra.run.smithi103.stdout:smithi139 172.21.15.139 2024-09-05T02:02:18.208 INFO:teuthology.orchestra.run.smithi103.stdout:smithi167 172.21.15.167 2024-09-05T02:02:18.208 INFO:teuthology.orchestra.run.smithi103.stdout:3 hosts in cluster 2024-09-05T02:02:18.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240a2c40 msgr2=0x7f70240a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:18.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240a2c40 0x7f70240a5100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70341993d0 tx=0x7f702000a2d0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 msgr2=0x7f7034197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:18.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 0x7f7034197eb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f702802a930 tx=0x7f7028003cd0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 shutdown_connections 2024-09-05T02:02:18.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f70240a2c40 0x7f70240a5100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70341026d0 0x7f703419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7034101d10 0x7f70341983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 --2- 172.21.15.103:0/1806625071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7034100b10 0x7f7034197eb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:18.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 >> 172.21.15.103:0/1806625071 conn(0x7f70340fc2a0 msgr2=0x7f703406a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:18.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 shutdown_connections 2024-09-05T02:02:18.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:18.210+0000 7f703ac50640 1 -- 172.21.15.103:0/1806625071 wait complete. 2024-09-05T02:02:18.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:18 smithi167 ceph-mon[35122]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:18.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:18 smithi167 ceph-mon[35122]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:18 smithi139 ceph-mon[34262]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:18 smithi139 ceph-mon[34262]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:18.955 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-05T02:02:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:19.459 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:19 smithi103 ceph-mon[30902]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:19.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:19 smithi167 ceph-mon[35122]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:19.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:19 smithi139 ceph-mon[34262]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.767+0000 7f067fc2b640 1 -- 172.21.15.103:0/2620777731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678107900 msgr2=0x7f0678109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.767+0000 7f067fc2b640 1 --2- 172.21.15.103:0/2620777731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678107900 0x7f0678109d50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0670009ca0 tx=0x7f067002a8c0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.767+0000 7f067fc2b640 1 -- 172.21.15.103:0/2620777731 shutdown_connections 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.767+0000 7f067fc2b640 1 --2- 172.21.15.103:0/2620777731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678107900 0x7f0678109d50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.767+0000 7f067fc2b640 1 --2- 172.21.15.103:0/2620777731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0678104fd0 0x7f06781073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.767+0000 7f067fc2b640 1 --2- 172.21.15.103:0/2620777731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06781026a0 0x7f0678104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.767+0000 7f067fc2b640 1 -- 172.21.15.103:0/2620777731 >> 172.21.15.103:0/2620777731 conn(0x7f06780fc2a0 msgr2=0x7f06780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.767+0000 7f067fc2b640 1 -- 172.21.15.103:0/2620777731 shutdown_connections 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.767+0000 7f067fc2b640 1 -- 172.21.15.103:0/2620777731 wait complete. 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067fc2b640 1 Processor -- start 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067fc2b640 1 -- start start 2024-09-05T02:02:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067fc2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06781026a0 0x7f0678197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067fc2b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678104fd0 0x7f0678198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067fc2b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0678107900 0x7f067819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067fc2b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f067810d840 con 0x7f0678104fd0 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067fc2b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f067810d6c0 con 0x7f06781026a0 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067fc2b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f067810d9c0 con 0x7f0678107900 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d1c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678104fd0 0x7f0678198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d1c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678104fd0 0x7f0678198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59084/0 (socket says 172.21.15.103:59084) 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d1c6640 1 -- 172.21.15.103:0/3935687955 learned_addr learned my addr 172.21.15.103:0/3935687955 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067e1c8640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0678107900 0x7f067819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d9c7640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06781026a0 0x7f0678197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d1c6640 1 -- 172.21.15.103:0/3935687955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0678107900 msgr2=0x7f067819f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d1c6640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0678107900 0x7f067819f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d1c6640 1 -- 172.21.15.103:0/3935687955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06781026a0 msgr2=0x7f0678197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d1c6640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06781026a0 0x7f0678197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d1c6640 1 -- 172.21.15.103:0/3935687955 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067819fb80 con 0x7f0678104fd0 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067e1c8640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0678107900 0x7f067819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d9c7640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06781026a0 0x7f0678197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.768+0000 7f067d1c6640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678104fd0 0x7f0678198490 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f066800b450 tx=0x7f066800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.769+0000 7f066effd640 1 -- 172.21.15.103:0/3935687955 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668015020 con 0x7f0678104fd0 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.769+0000 7f066effd640 1 -- 172.21.15.103:0/3935687955 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0668011d40 con 0x7f0678104fd0 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.769+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067819fe70 con 0x7f0678104fd0 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.769+0000 7f066effd640 1 -- 172.21.15.103:0/3935687955 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066800f340 con 0x7f0678104fd0 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.769+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06781a03a0 con 0x7f0678104fd0 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.770+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06780fdd90 con 0x7f0678104fd0 2024-09-05T02:02:19.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.771+0000 7f066effd640 1 -- 172.21.15.103:0/3935687955 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0668018020 con 0x7f0678104fd0 2024-09-05T02:02:19.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.772+0000 7f066effd640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06640a2c40 0x7f06640a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:19.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.772+0000 7f067d9c7640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06640a2c40 0x7f06640a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:19.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.772+0000 7f066effd640 1 -- 172.21.15.103:0/3935687955 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f06680cef10 con 0x7f0678104fd0 2024-09-05T02:02:19.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.772+0000 7f067d9c7640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06640a2c40 0x7f06640a5100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0674006fd0 tx=0x7f0674008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:19.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.776+0000 7f066effd640 1 -- 172.21.15.103:0/3935687955 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f066801f020 con 0x7f0678104fd0 2024-09-05T02:02:19.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.960+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0678062630 con 0x7f06640a2c40 2024-09-05T02:02:19.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.962+0000 7f066effd640 1 -- 172.21.15.103:0/3935687955 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0678062630 con 0x7f06640a2c40 2024-09-05T02:02:19.963 INFO:teuthology.orchestra.run.smithi103.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-05T02:02:19.963 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T02:02:19.963 INFO:teuthology.orchestra.run.smithi103.stdout:smithi139 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N6400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T02:02:19.963 INFO:teuthology.orchestra.run.smithi103.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T02:02:19.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06640a2c40 msgr2=0x7f06640a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:19.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06640a2c40 0x7f06640a5100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0674006fd0 tx=0x7f0674008040 comp rx=0 tx=0).stop 2024-09-05T02:02:19.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678104fd0 msgr2=0x7f0678198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:19.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678104fd0 0x7f0678198490 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f066800b450 tx=0x7f066800b910 comp rx=0 tx=0).stop 2024-09-05T02:02:19.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 shutdown_connections 2024-09-05T02:02:19.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f06640a2c40 0x7f06640a5100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0678107900 0x7f067819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0678104fd0 0x7f0678198490 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 --2- 172.21.15.103:0/3935687955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06781026a0 0x7f0678197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:19.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 >> 172.21.15.103:0/3935687955 conn(0x7f06780fc2a0 msgr2=0x7f0678103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:19.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.964+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 shutdown_connections 2024-09-05T02:02:19.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:19.965+0000 7f067fc2b640 1 -- 172.21.15.103:0/3935687955 wait complete. 2024-09-05T02:02:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:20 smithi167 ceph-mon[35122]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:02:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:20 smithi139 ceph-mon[34262]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:02:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:20 smithi103 ceph-mon[30902]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T02:02:20.684 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T02:02:20.693 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-05T02:02:20.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-05T02:02:20.939 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:21.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:21 smithi103 ceph-mon[30902]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:21.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.430+0000 7fdb2a930640 1 -- 172.21.15.103:0/3537155703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24102700 msgr2=0x7fdb24104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:21.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.430+0000 7fdb2a930640 1 --2- 172.21.15.103:0/3537155703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24102700 0x7fdb24104af0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009900 tx=0x7fdb1402a410 comp rx=0 tx=0).stop 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.430+0000 7fdb2a930640 1 -- 172.21.15.103:0/3537155703 shutdown_connections 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.430+0000 7fdb2a930640 1 --2- 172.21.15.103:0/3537155703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb24107960 0x7fdb24109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.430+0000 7fdb2a930640 1 --2- 172.21.15.103:0/3537155703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24105030 0x7fdb24107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.430+0000 7fdb2a930640 1 --2- 172.21.15.103:0/3537155703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24102700 0x7fdb24104af0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.430+0000 7fdb2a930640 1 -- 172.21.15.103:0/3537155703 >> 172.21.15.103:0/3537155703 conn(0x7fdb240fc280 msgr2=0x7fdb240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.430+0000 7fdb2a930640 1 -- 172.21.15.103:0/3537155703 shutdown_connections 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.430+0000 7fdb2a930640 1 -- 172.21.15.103:0/3537155703 wait complete. 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb2a930640 1 Processor -- start 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb2a930640 1 -- start start 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb2a930640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb24102700 0x7fdb24102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb2a930640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24105030 0x7fdb241006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb2a930640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24107960 0x7fdb24100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:21.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb2a930640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2410d710 con 0x7fdb24105030 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb2a930640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2410d590 con 0x7fdb24102700 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb2a930640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2410d890 con 0x7fdb24107960 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24105030 0x7fdb241006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24105030 0x7fdb241006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59100/0 (socket says 172.21.15.103:59100) 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb237fe640 1 -- 172.21.15.103:0/694860663 learned_addr learned my addr 172.21.15.103:0/694860663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb23fff640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb24102700 0x7fdb24102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb28ecd640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24107960 0x7fdb24100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb237fe640 1 -- 172.21.15.103:0/694860663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24107960 msgr2=0x7fdb24100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb237fe640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24107960 0x7fdb24100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb237fe640 1 -- 172.21.15.103:0/694860663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb24102700 msgr2=0x7fdb24102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb237fe640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb24102700 0x7fdb24102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.431+0000 7fdb237fe640 1 -- 172.21.15.103:0/694860663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb241014b0 con 0x7fdb24105030 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.432+0000 7fdb237fe640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24105030 0x7fdb241006b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdb18005fd0 tx=0x7fdb1800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.432+0000 7fdb217fa640 1 -- 172.21.15.103:0/694860663 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18017070 con 0x7fdb24105030 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.432+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb241a1fe0 con 0x7fdb24105030 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.432+0000 7fdb217fa640 1 -- 172.21.15.103:0/694860663 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb18009c30 con 0x7fdb24105030 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.432+0000 7fdb217fa640 1 -- 172.21.15.103:0/694860663 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18003d40 con 0x7fdb24105030 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.432+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb241a2440 con 0x7fdb24105030 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.433+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb240fdd70 con 0x7fdb24105030 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.434+0000 7fdb217fa640 1 -- 172.21.15.103:0/694860663 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdb18003ee0 con 0x7fdb24105030 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.435+0000 7fdb217fa640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb0c0a2c10 0x7fdb0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.435+0000 7fdb23fff640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb0c0a2c10 0x7fdb0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.435+0000 7fdb217fa640 1 -- 172.21.15.103:0/694860663 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fdb180cf930 con 0x7fdb24105030 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.435+0000 7fdb23fff640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb0c0a2c10 0x7fdb0c0a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006010 tx=0x7fdb14000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:21.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.438+0000 7fdb217fa640 1 -- 172.21.15.103:0/694860663 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb18020030 con 0x7fdb24105030 2024-09-05T02:02:21.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:21 smithi167 ceph-mon[35122]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:21.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:21 smithi139 ceph-mon[34262]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T02:02:21.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.647+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fdb2402d9c0 con 0x7fdb24105030 2024-09-05T02:02:21.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.647+0000 7fdb217fa640 1 -- 172.21.15.103:0/694860663 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3146 (secure 0 0 0) 0x7fdb1809c3d0 con 0x7fdb24105030 2024-09-05T02:02:21.648 INFO:teuthology.orchestra.run.smithi103.stdout:e28 2024-09-05T02:02:21.648 INFO:teuthology.orchestra.run.smithi103.stdout:btime 2024-09-05T02:00:59:232233+0000 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:legacy client fscid: 1 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:Filesystem 'cephfs' (1) 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:fs_name cephfs 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 28 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-05T02:00:15.086875+0000 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-05T02:00:59.232218+0000 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:tableserver 0 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:root 0 2024-09-05T02:02:21.649 INFO:teuthology.orchestra.run.smithi103.stdout:session_timeout 300 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:session_autoclose 300 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:max_file_size 1099511627776 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:max_xattr_size 65536 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:required_client_features {} 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure 0 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure_osd_epoch 0 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:max_mds 5 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:in 0,1,2,3,4 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:up {0=24497,1=24466,2=24464,3=24457,4=24449} 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:failed 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:damaged 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:stopped 2024-09-05T02:02:21.650 INFO:teuthology.orchestra.run.smithi103.stdout:data_pools [3,4] 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:metadata_pool 2 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:inline_data disabled 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:balancer 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:bal_rank_mask -1 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:standby_count_wanted 0 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:qdb_cluster leader: 24497 members: 24497,24466,24464,24457,24449 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b{0:24497} state up:active seq 4 addr [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.f{0:24445} state up:standby-replay seq 1 addr [v2:172.21.15.167:6834/1110459707,v1:172.21.15.167:6835/1110459707] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.i{1:24466} state up:active seq 17 addr [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.l{1:24430} state up:standby-replay seq 1 addr [v2:172.21.15.167:6832/2095779466,v1:172.21.15.167:6833/2095779466] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.h{2:24464} state up:active seq 18 addr [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.k{2:24422} state up:standby-replay seq 1 addr [v2:172.21.15.139:6832/4042514104,v1:172.21.15.139:6833/4042514104] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c{3:24457} state up:active seq 19 addr [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.651 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a{3:14625} state up:standby-replay seq 1 addr [v2:172.21.15.103:6840/426074448,v1:172.21.15.103:6841/426074448] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.652 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.e{4:24449} state up:active seq 20 addr [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.652 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.103:6838/1603518036,v1:172.21.15.103:6839/1603518036] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.652 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:21.652 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:21.652 INFO:teuthology.orchestra.run.smithi103.stdout:Standby daemons: 2024-09-05T02:02:21.652 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:21.652 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.103:6834/726289988,v1:172.21.15.103:6835/726289988] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.652 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.103:6836/2025990895,v1:172.21.15.103:6837/2025990895] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T02:02:21.653 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 28 2024-09-05T02:02:21.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.649+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb0c0a2c10 msgr2=0x7fdb0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:21.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.649+0000 7fdb2a930640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb0c0a2c10 0x7fdb0c0a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006010 tx=0x7fdb14000bc0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.649+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24105030 msgr2=0x7fdb241006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:21.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.649+0000 7fdb2a930640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24105030 0x7fdb241006b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdb18005fd0 tx=0x7fdb1800dde0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.650+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 shutdown_connections 2024-09-05T02:02:21.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.650+0000 7fdb2a930640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fdb0c0a2c10 0x7fdb0c0a50d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.650+0000 7fdb2a930640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24107960 0x7fdb24100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.650+0000 7fdb2a930640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb24105030 0x7fdb241006b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.650+0000 7fdb2a930640 1 --2- 172.21.15.103:0/694860663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb24102700 0x7fdb24102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:21.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.650+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 >> 172.21.15.103:0/694860663 conn(0x7fdb240fc280 msgr2=0x7fdb241031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:21.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.650+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 shutdown_connections 2024-09-05T02:02:21.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:21.650+0000 7fdb2a930640 1 -- 172.21.15.103:0/694860663 wait complete. 2024-09-05T02:02:22.344 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:22 smithi103 ceph-mon[30902]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:02:22.344 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:22 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/694860663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T02:02:22.383 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-05T02:02:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:22 smithi167 ceph-mon[35122]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:02:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:22 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/694860663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T02:02:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:22 smithi139 ceph-mon[34262]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T02:02:22.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:22 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/694860663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T02:02:23.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.136+0000 7ff2d3752640 1 -- 172.21.15.103:0/1030735564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc100b10 msgr2=0x7ff2cc100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:23.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.136+0000 7ff2d3752640 1 --2- 172.21.15.103:0/1030735564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc100b10 0x7ff2cc100f10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009900 tx=0x7ff2c802a410 comp rx=0 tx=0).stop 2024-09-05T02:02:23.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 -- 172.21.15.103:0/1030735564 shutdown_connections 2024-09-05T02:02:23.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 --2- 172.21.15.103:0/1030735564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2cc1026d0 0x7ff2cc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 --2- 172.21.15.103:0/1030735564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2cc101d10 0x7ff2cc102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 --2- 172.21.15.103:0/1030735564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc100b10 0x7ff2cc100f10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 -- 172.21.15.103:0/1030735564 >> 172.21.15.103:0/1030735564 conn(0x7ff2cc0fc2a0 msgr2=0x7ff2cc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 -- 172.21.15.103:0/1030735564 shutdown_connections 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 -- 172.21.15.103:0/1030735564 wait complete. 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 Processor -- start 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 -- start start 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2cc100b10 0x7ff2cc06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc101d10 0x7ff2cc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2cc1026d0 0x7ff2cc19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.137+0000 7ff2d3752640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc10c720 con 0x7ff2cc101d10 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d3752640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc10c5a0 con 0x7ff2cc1026d0 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d3752640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc10c8a0 con 0x7ff2cc100b10 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d0ced640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc101d10 0x7ff2cc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d0ced640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc101d10 0x7ff2cc19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59120/0 (socket says 172.21.15.103:59120) 2024-09-05T02:02:23.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d0ced640 1 -- 172.21.15.103:0/383312891 learned_addr learned my addr 172.21.15.103:0/383312891 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d14ee640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2cc100b10 0x7ff2cc06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d0ced640 1 -- 172.21.15.103:0/383312891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2cc100b10 msgr2=0x7ff2cc06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d0ced640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2cc100b10 0x7ff2cc06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d0ced640 1 -- 172.21.15.103:0/383312891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2cc1026d0 msgr2=0x7ff2cc19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d1cef640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2cc1026d0 0x7ff2cc19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d0ced640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2cc1026d0 0x7ff2cc19f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d0ced640 1 -- 172.21.15.103:0/383312891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc19fc40 con 0x7ff2cc101d10 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d0ced640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc101d10 0x7ff2cc19cd10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00b4a0 tx=0x7ff2bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2c27fc640 1 -- 172.21.15.103:0/383312891 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc014070 con 0x7ff2cc101d10 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2c27fc640 1 -- 172.21.15.103:0/383312891 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2bc010ac0 con 0x7ff2cc101d10 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2cc19fed0 con 0x7ff2cc101d10 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2c27fc640 1 -- 172.21.15.103:0/383312891 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc017c30 con 0x7ff2cc101d10 2024-09-05T02:02:23.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.138+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2cc1a0400 con 0x7ff2cc101d10 2024-09-05T02:02:23.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.139+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2cc108040 con 0x7ff2cc101d10 2024-09-05T02:02:23.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.140+0000 7ff2c27fc640 1 -- 172.21.15.103:0/383312891 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff2bc017070 con 0x7ff2cc101d10 2024-09-05T02:02:23.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.141+0000 7ff2c27fc640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff2b80a2c10 0x7ff2b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:23.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.141+0000 7ff2d14ee640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff2b80a2c10 0x7ff2b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:23.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.141+0000 7ff2c27fc640 1 -- 172.21.15.103:0/383312891 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7ff2bc0ceb90 con 0x7ff2cc101d10 2024-09-05T02:02:23.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.141+0000 7ff2d14ee640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff2b80a2c10 0x7ff2b80a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8000c00 tx=0x7ff2c80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:23.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.144+0000 7ff2c27fc640 1 -- 172.21.15.103:0/383312891 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2bc09b740 con 0x7ff2cc101d10 2024-09-05T02:02:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.314+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff2cc02d9c0 con 0x7ff2cc101d10 2024-09-05T02:02:23.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.315+0000 7ff2c27fc640 1 -- 172.21.15.103:0/383312891 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5621 (secure 0 0 0) 0x7ff2bc09b040 con 0x7ff2cc101d10 2024-09-05T02:02:23.315 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 88 2024-09-05T02:02:23.315 INFO:teuthology.orchestra.run.smithi103.stdout:fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-05T01:52:29.045755+0000 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-05T02:01:26.255904+0000 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:crush_version 27 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:full_ratio 0.95 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:backfillfull_ratio 0.9 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:nearfull_ratio 0.85 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:require_min_compat_client luminous 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:min_compat_client luminous 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:require_osd_release squid 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:stretch_mode_enabled false 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-05T02:02:23.316 INFO:teuthology.orchestra.run.smithi103.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:max_osd 12 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6802/2389557969,v1:172.21.15.103:6803/2389557969] [v2:172.21.15.103:6804/2389557969,v1:172.21.15.103:6805/2389557969] exists,up 6fe0c785-c54d-4779-8255-8d88e39aa32d 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6800/491231014,v1:172.21.15.139:6801/491231014] [v2:172.21.15.139:6802/491231014,v1:172.21.15.139:6803/491231014] exists,up e0815a57-5a14-4d03-9045-36fd3649ab31 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 up in weight 1 up_from 17 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/945108637,v1:172.21.15.167:6801/945108637] [v2:172.21.15.167:6802/945108637,v1:172.21.15.167:6803/945108637] exists,up 13eeb8f3-8b22-415b-abd9-d8d054bc1093 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 up in weight 1 up_from 24 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6810/845945713,v1:172.21.15.103:6811/845945713] [v2:172.21.15.103:6812/845945713,v1:172.21.15.103:6813/845945713] exists,up 80e9c6a8-bf1d-45a1-9e94-6abba2659c17 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 up in weight 1 up_from 31 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6808/2565711850,v1:172.21.15.139:6809/2565711850] [v2:172.21.15.139:6810/2565711850,v1:172.21.15.139:6811/2565711850] exists,up ed03c495-8ced-4b29-af59-3fec3dab74fb 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 up in weight 1 up_from 36 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/4050841663,v1:172.21.15.167:6809/4050841663] [v2:172.21.15.167:6810/4050841663,v1:172.21.15.167:6811/4050841663] exists,up 37eca6d4-b597-41fc-af54-5db67c9a4d57 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 up in weight 1 up_from 43 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6818/2808952554,v1:172.21.15.103:6819/2808952554] [v2:172.21.15.103:6820/2808952554,v1:172.21.15.103:6821/2808952554] exists,up 82343e71-6c0c-4ea3-b050-23b2bcae9e50 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 up in weight 1 up_from 48 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6816/3442676613,v1:172.21.15.139:6817/3442676613] [v2:172.21.15.139:6818/3442676613,v1:172.21.15.139:6819/3442676613] exists,up c96851b2-9ad0-4adb-b840-9bd11ab6655e 2024-09-05T02:02:23.317 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 up in weight 1 up_from 55 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/639772211,v1:172.21.15.167:6817/639772211] [v2:172.21.15.167:6818/639772211,v1:172.21.15.167:6819/639772211] exists,up f5b69841-792a-4d17-8406-278c8d11a20a 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 up in weight 1 up_from 62 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6826/3249722651,v1:172.21.15.103:6827/3249722651] [v2:172.21.15.103:6828/3249722651,v1:172.21.15.103:6829/3249722651] exists,up 0e609bbd-4a74-489f-8b8b-47bb13175b8d 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 up in weight 1 up_from 67 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6824/3256384109,v1:172.21.15.139:6825/3256384109] [v2:172.21.15.139:6826/3256384109,v1:172.21.15.139:6827/3256384109] exists,up 52a68fc4-9a55-4a43-a1da-72019facc4c9 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 up in weight 1 up_from 74 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/259212791,v1:172.21.15.167:6825/259212791] [v2:172.21.15.167:6826/259212791,v1:172.21.15.167:6827/259212791] exists,up 356aa3c1-baca-473e-a6fc-cf366d4783a0 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.10 [2,4] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.0 [4,6] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.10 [5,9] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.17 [4,8] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.22 [3,9] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.3c [4,9] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 4.6 [9,11] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 4.b [9,8] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 4.12 [5,10] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 4.13 [9,8] 2024-09-05T02:02:23.318 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/2384335573 expires 2024-09-06T01:53:25.001822+0000 2024-09-05T02:02:23.319 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/2384335573 expires 2024-09-06T01:53:25.001822+0000 2024-09-05T02:02:23.319 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/2155492495 expires 2024-09-06T01:53:25.001822+0000 2024-09-05T02:02:23.319 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/396485172 expires 2024-09-06T01:53:25.001822+0000 2024-09-05T02:02:23.319 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1467689006 expires 2024-09-06T01:53:25.001822+0000 2024-09-05T02:02:23.319 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/3874104813 expires 2024-09-06T01:53:03.000202+0000 2024-09-05T02:02:23.319 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1573405851 expires 2024-09-06T01:53:03.000202+0000 2024-09-05T02:02:23.319 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/2132077607 expires 2024-09-06T01:53:03.000202+0000 2024-09-05T02:02:23.319 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/3892241075 expires 2024-09-06T01:53:03.000202+0000 2024-09-05T02:02:23.319 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/2132077607 expires 2024-09-06T01:53:03.000202+0000 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff2b80a2c10 msgr2=0x7ff2b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff2b80a2c10 0x7ff2b80a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8000c00 tx=0x7ff2c80314a0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc101d10 msgr2=0x7ff2cc19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc101d10 0x7ff2cc19cd10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00b4a0 tx=0x7ff2bc00b960 comp rx=0 tx=0).stop 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 shutdown_connections 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2cc1026d0 0x7ff2cc19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2cc101d10 0x7ff2cc19cd10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff2b80a2c10 0x7ff2b80a50d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 --2- 172.21.15.103:0/383312891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2cc100b10 0x7ff2cc06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 >> 172.21.15.103:0/383312891 conn(0x7ff2cc0fc2a0 msgr2=0x7ff2cc105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:23.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 shutdown_connections 2024-09-05T02:02:23.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:23.317+0000 7ff2d3752640 1 -- 172.21.15.103:0/383312891 wait complete. 2024-09-05T02:02:23.993 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-05T02:02:24.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:02:24.784 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:24 smithi139 ceph-mon[34262]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:24.784 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:24 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/383312891' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T02:02:24.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:24 smithi167 ceph-mon[35122]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:24.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:24 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/383312891' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T02:02:24.784 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:24 smithi103 ceph-mon[30902]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:02:24.784 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:24 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/383312891' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T02:02:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:25.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.579+0000 7f53176a2640 1 -- 172.21.15.103:0/1268888064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 msgr2=0x7f531010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.579+0000 7f53176a2640 1 --2- 172.21.15.103:0/1268888064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 0x7f531010d950 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5308009cd0 tx=0x7f530802a8e0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.579+0000 7f53176a2640 1 -- 172.21.15.103:0/1268888064 shutdown_connections 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.579+0000 7f53176a2640 1 --2- 172.21.15.103:0/1268888064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 0x7f531010d950 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.579+0000 7f53176a2640 1 --2- 172.21.15.103:0/1268888064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310100a80 0x7f5310100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.579+0000 7f53176a2640 1 --2- 172.21.15.103:0/1268888064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5310106a80 0x7f5310106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.579+0000 7f53176a2640 1 -- 172.21.15.103:0/1268888064 >> 172.21.15.103:0/1268888064 conn(0x7f53100fc7a0 msgr2=0x7f53100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.579+0000 7f53176a2640 1 -- 172.21.15.103:0/1268888064 shutdown_connections 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.579+0000 7f53176a2640 1 -- 172.21.15.103:0/1268888064 wait complete. 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f53176a2640 1 Processor -- start 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f53176a2640 1 -- start start 2024-09-05T02:02:25.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f53176a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5310100a80 0x7f53100fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f53176a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 0x7f53100fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f53176a2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310106a80 0x7f53100fd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f53176a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5310110d30 con 0x7f5310101420 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f53176a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5310110bb0 con 0x7f5310100a80 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f53176a2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5310110eb0 con 0x7f5310106a80 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f5314c3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 0x7f53100fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f5314c3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 0x7f53100fd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59142/0 (socket says 172.21.15.103:59142) 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f5314c3d640 1 -- 172.21.15.103:0/3447318473 learned_addr learned my addr 172.21.15.103:0/3447318473 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f531543e640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5310100a80 0x7f53100fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f5315c3f640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310106a80 0x7f53100fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f5314c3d640 1 -- 172.21.15.103:0/3447318473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310106a80 msgr2=0x7f53100fd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f5314c3d640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310106a80 0x7f53100fd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f5314c3d640 1 -- 172.21.15.103:0/3447318473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5310100a80 msgr2=0x7f53100fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f5314c3d640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5310100a80 0x7f53100fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.580+0000 7f5314c3d640 1 -- 172.21.15.103:0/3447318473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53101a67e0 con 0x7f5310101420 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.581+0000 7f5314c3d640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 0x7f53100fd2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f530000d770 tx=0x7f530000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.581+0000 7f53067fc640 1 -- 172.21.15.103:0/3447318473 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300017070 con 0x7f5310101420 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.581+0000 7f53067fc640 1 -- 172.21.15.103:0/3447318473 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53000054d0 con 0x7f5310101420 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.581+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53101a6ad0 con 0x7f5310101420 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.581+0000 7f53067fc640 1 -- 172.21.15.103:0/3447318473 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300004cf0 con 0x7f5310101420 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.581+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53101a6fc0 con 0x7f5310101420 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.582+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5310100ee0 con 0x7f5310101420 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.583+0000 7f53067fc640 1 -- 172.21.15.103:0/3447318473 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5300004e90 con 0x7f5310101420 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.584+0000 7f53067fc640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.584+0000 7f531543e640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:25.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.584+0000 7f53067fc640 1 -- 172.21.15.103:0/3447318473 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f53000ceef0 con 0x7f5310101420 2024-09-05T02:02:25.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.584+0000 7f531543e640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f530c00bb60 tx=0x7f530c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:25.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.588+0000 7f53067fc640 1 -- 172.21.15.103:0/3447318473 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f530009b340 con 0x7f5310101420 2024-09-05T02:02:25.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.791+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f531010a050 con 0x7f5310101420 2024-09-05T02:02:25.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.792+0000 7f53067fc640 1 -- 172.21.15.103:0/3447318473 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7f530009f640 con 0x7f5310101420 2024-09-05T02:02:25.793 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:25.793 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":28,"btime":"2024-09-05T02:00:59:232233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:00:59.232218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:02:25.793 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 28 2024-09-05T02:02:25.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f52fc0a2c10 msgr2=0x7f52fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:25.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f530c00bb60 tx=0x7f530c00b390 comp rx=0 tx=0).stop 2024-09-05T02:02:25.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 msgr2=0x7f53100fd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:25.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 0x7f53100fd2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f530000d770 tx=0x7f530000dc30 comp rx=0 tx=0).stop 2024-09-05T02:02:25.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 shutdown_connections 2024-09-05T02:02:25.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310106a80 0x7f53100fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310101420 0x7f53100fd2f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 --2- 172.21.15.103:0/3447318473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5310100a80 0x7f53100fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:25.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 >> 172.21.15.103:0/3447318473 conn(0x7f53100fc7a0 msgr2=0x7f531006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:25.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 shutdown_connections 2024-09-05T02:02:25.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:25.794+0000 7f53176a2640 1 -- 172.21.15.103:0/3447318473 wait complete. 2024-09-05T02:02:26.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:26 smithi103 ceph-mon[30902]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:02:26.481 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-05T02:02:26.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-05T02:02:26.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-05T02:02:26.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:26 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3447318473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:26.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:02:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:26 smithi167 ceph-mon[35122]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:02:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-05T02:02:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-05T02:02:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-05T02:02:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:26 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3447318473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:26.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:26 smithi139 ceph-mon[34262]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T02:02:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-05T02:02:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-05T02:02:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-05T02:02:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:26 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3447318473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:27.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.232+0000 7fa1729b1640 1 -- 172.21.15.103:0/1119123393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 msgr2=0x7fa16c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:27.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.232+0000 7fa1729b1640 1 --2- 172.21.15.103:0/1119123393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 0x7fa16c10d950 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa164009cd0 tx=0x7fa16402a910 comp rx=0 tx=0).stop 2024-09-05T02:02:27.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.232+0000 7fa1729b1640 1 -- 172.21.15.103:0/1119123393 shutdown_connections 2024-09-05T02:02:27.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.232+0000 7fa1729b1640 1 --2- 172.21.15.103:0/1119123393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 0x7fa16c10d950 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.232+0000 7fa1729b1640 1 --2- 172.21.15.103:0/1119123393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa16c100a80 0x7fa16c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.232+0000 7fa1729b1640 1 --2- 172.21.15.103:0/1119123393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa16c106a80 0x7fa16c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.232+0000 7fa1729b1640 1 -- 172.21.15.103:0/1119123393 >> 172.21.15.103:0/1119123393 conn(0x7fa16c0fc7a0 msgr2=0x7fa16c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:27.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.232+0000 7fa1729b1640 1 -- 172.21.15.103:0/1119123393 shutdown_connections 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.232+0000 7fa1729b1640 1 -- 172.21.15.103:0/1119123393 wait complete. 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa1729b1640 1 Processor -- start 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa1729b1640 1 -- start start 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa1729b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa16c100a80 0x7fa16c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa1729b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 0x7fa16c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa1729b1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa16c106a80 0x7fa16c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa1729b1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c111ef0 con 0x7fa16c101420 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa1729b1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c111d70 con 0x7fa16c100a80 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa1729b1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c112070 con 0x7fa16c106a80 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa163fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 0x7fa16c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:27.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa163fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 0x7fa16c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59160/0 (socket says 172.21.15.103:59160) 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa163fff640 1 -- 172.21.15.103:0/3028235111 learned_addr learned my addr 172.21.15.103:0/3028235111 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa163fff640 1 -- 172.21.15.103:0/3028235111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa16c106a80 msgr2=0x7fa16c19cd90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa163fff640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa16c106a80 0x7fa16c19cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa163fff640 1 -- 172.21.15.103:0/3028235111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa16c100a80 msgr2=0x7fa16c1984c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa163fff640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa16c100a80 0x7fa16c1984c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.233+0000 7fa163fff640 1 -- 172.21.15.103:0/3028235111 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c19d470 con 0x7fa16c101420 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.234+0000 7fa163fff640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 0x7fa16c198a00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa15400b4a0 tx=0x7fa15400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.234+0000 7fa169ffb640 1 -- 172.21.15.103:0/3028235111 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154014070 con 0x7fa16c101420 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.234+0000 7fa169ffb640 1 -- 172.21.15.103:0/3028235111 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa154010ac0 con 0x7fa16c101420 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.234+0000 7fa169ffb640 1 -- 172.21.15.103:0/3028235111 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154017c30 con 0x7fa16c101420 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.234+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16c19d760 con 0x7fa16c101420 2024-09-05T02:02:27.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.234+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16c1a5040 con 0x7fa16c101420 2024-09-05T02:02:27.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.235+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa16c100f60 con 0x7fa16c101420 2024-09-05T02:02:27.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.236+0000 7fa169ffb640 1 -- 172.21.15.103:0/3028235111 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa154017070 con 0x7fa16c101420 2024-09-05T02:02:27.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.236+0000 7fa169ffb640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1580a2b80 0x7fa1580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:27.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.237+0000 7fa16bfff640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1580a2b80 0x7fa1580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:27.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.237+0000 7fa169ffb640 1 -- 172.21.15.103:0/3028235111 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa1540ce890 con 0x7fa16c101420 2024-09-05T02:02:27.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.237+0000 7fa16bfff640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1580a2b80 0x7fa1580a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009670 tx=0x7fa15c006bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:27.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.241+0000 7fa169ffb640 1 -- 172.21.15.103:0/3028235111 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa15409b440 con 0x7fa16c101420 2024-09-05T02:02:27.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.444+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa16c02d7c0 con 0x7fa16c101420 2024-09-05T02:02:27.445 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-05T02:02:27.445 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-05T02:02:27.445 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:27 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-05T02:02:27.445 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:27 smithi103 ceph-mon[30902]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T02:02:27.445 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:27 smithi103 ceph-mon[30902]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T02:02:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.444+0000 7fa169ffb640 1 -- 172.21.15.103:0/3028235111 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7fa15409ad40 con 0x7fa16c101420 2024-09-05T02:02:27.446 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:27.446 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":28,"btime":"2024-09-05T02:00:59:232233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:00:59.232218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:02:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 28 2024-09-05T02:02:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1580a2b80 msgr2=0x7fa1580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1580a2b80 0x7fa1580a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009670 tx=0x7fa15c006bd0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 msgr2=0x7fa16c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 0x7fa16c198a00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa15400b4a0 tx=0x7fa15400b960 comp rx=0 tx=0).stop 2024-09-05T02:02:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 shutdown_connections 2024-09-05T02:02:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa1580a2b80 0x7fa1580a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa16c106a80 0x7fa16c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa16c101420 0x7fa16c198a00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 --2- 172.21.15.103:0/3028235111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa16c100a80 0x7fa16c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 >> 172.21.15.103:0/3028235111 conn(0x7fa16c0fc7a0 msgr2=0x7fa16c0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.446+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 shutdown_connections 2024-09-05T02:02:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:27.447+0000 7fa1729b1640 1 -- 172.21.15.103:0/3028235111 wait complete. 2024-09-05T02:02:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-05T02:02:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-05T02:02:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:27 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-05T02:02:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:27 smithi167 ceph-mon[35122]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T02:02:27.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:27 smithi167 ceph-mon[35122]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T02:02:27.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-05T02:02:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-05T02:02:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:27 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-05T02:02:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:27 smithi139 ceph-mon[34262]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T02:02:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:27 smithi139 ceph-mon[34262]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T02:02:28.171 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:02:28.419 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:28.540 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:28 smithi103 ceph-mon[30902]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-05T02:02:28.540 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:28 smithi103 ceph-mon[30902]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T02:02:28.540 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:28 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3028235111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:28 smithi167 ceph-mon[35122]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-05T02:02:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:28 smithi167 ceph-mon[35122]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T02:02:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:28 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3028235111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:28.648 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:28 smithi139 ceph-mon[34262]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-05T02:02:28.648 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:28 smithi139 ceph-mon[34262]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T02:02:28.648 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:28 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3028235111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:28.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 -- 172.21.15.103:0/135631431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928101420 msgr2=0x7fa92810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:28.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/135631431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928101420 0x7fa92810d950 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa920009930 tx=0x7fa92002a420 comp rx=0 tx=0).stop 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 -- 172.21.15.103:0/135631431 shutdown_connections 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/135631431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928101420 0x7fa92810d950 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/135631431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa928100a80 0x7fa928100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/135631431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa928106a80 0x7fa928106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 -- 172.21.15.103:0/135631431 >> 172.21.15.103:0/135631431 conn(0x7fa9280fc7a0 msgr2=0x7fa9280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 -- 172.21.15.103:0/135631431 shutdown_connections 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 -- 172.21.15.103:0/135631431 wait complete. 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 Processor -- start 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 -- start start 2024-09-05T02:02:28.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928100a80 0x7fa928199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.890+0000 7fa92ed2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa928101420 0x7fa9281996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92ed2b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa928106a80 0x7fa9281a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92ed2b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa928110d30 con 0x7fa928100a80 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92ed2b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa928110bb0 con 0x7fa928101420 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92ed2b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa928110eb0 con 0x7fa928106a80 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92cac7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928100a80 0x7fa928199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92cac7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928100a80 0x7fa928199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59164/0 (socket says 172.21.15.103:59164) 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92cac7640 1 -- 172.21.15.103:0/3407834264 learned_addr learned my addr 172.21.15.103:0/3407834264 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa927fff640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa928101420 0x7fa9281996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92cac7640 1 -- 172.21.15.103:0/3407834264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa928106a80 msgr2=0x7fa9281a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92cac7640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa928106a80 0x7fa9281a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:28.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92cac7640 1 -- 172.21.15.103:0/3407834264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa928101420 msgr2=0x7fa9281996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92cac7640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa928101420 0x7fa9281996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92cac7640 1 -- 172.21.15.103:0/3407834264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9281a4720 con 0x7fa928100a80 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92cac7640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928100a80 0x7fa928199160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa91000d950 tx=0x7fa91000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa925ffb640 1 -- 172.21.15.103:0/3407834264 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910017070 con 0x7fa928100a80 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa925ffb640 1 -- 172.21.15.103:0/3407834264 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa910009c30 con 0x7fa928100a80 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa925ffb640 1 -- 172.21.15.103:0/3407834264 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91000f620 con 0x7fa928100a80 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.891+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9281a48f0 con 0x7fa928100a80 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.892+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9281a4e80 con 0x7fa928100a80 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.892+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa928100ee0 con 0x7fa928100a80 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.893+0000 7fa925ffb640 1 -- 172.21.15.103:0/3407834264 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa9100109a0 con 0x7fa928100a80 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.894+0000 7fa925ffb640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa9140a2c00 0x7fa9140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.894+0000 7fa925ffb640 1 -- 172.21.15.103:0/3407834264 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa9100cef20 con 0x7fa928100a80 2024-09-05T02:02:28.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.894+0000 7fa927fff640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa9140a2c00 0x7fa9140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:28.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.894+0000 7fa927fff640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa9140a2c00 0x7fa9140a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa92806b320 tx=0x7fa918009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:28.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:28.898+0000 7fa925ffb640 1 -- 172.21.15.103:0/3407834264 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa910009de0 con 0x7fa928100a80 2024-09-05T02:02:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.065+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa928108a80 con 0x7fa928100a80 2024-09-05T02:02:29.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.067+0000 7fa925ffb640 1 -- 172.21.15.103:0/3407834264 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24926 (secure 0 0 0) 0x7fa91009bad0 con 0x7fa928100a80 2024-09-05T02:02:29.068 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:29.068 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":90,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:02:27.192641+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:02:29.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa9140a2c00 msgr2=0x7fa9140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:29.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa9140a2c00 0x7fa9140a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa92806b320 tx=0x7fa918009680 comp rx=0 tx=0).stop 2024-09-05T02:02:29.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928100a80 msgr2=0x7fa928199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:29.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928100a80 0x7fa928199160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa91000d950 tx=0x7fa91000de10 comp rx=0 tx=0).stop 2024-09-05T02:02:29.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 shutdown_connections 2024-09-05T02:02:29.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa9140a2c00 0x7fa9140a50c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:29.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa928106a80 0x7fa9281a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:29.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa928101420 0x7fa9281996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:29.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 --2- 172.21.15.103:0/3407834264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa928100a80 0x7fa928199160 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:29.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.069+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 >> 172.21.15.103:0/3407834264 conn(0x7fa9280fc7a0 msgr2=0x7fa9280fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:29.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.070+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 shutdown_connections 2024-09-05T02:02:29.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:29.070+0000 7fa92ed2b640 1 -- 172.21.15.103:0/3407834264 wait complete. 2024-09-05T02:02:29.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:29 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3407834264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:02:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:29 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3407834264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:02:29.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:29 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3407834264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:02:29.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T02:00:15.086875+0000', 'modified': '2024-09-05T02:00:59.232218+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24497, 'mds_1': 24466, 'mds_2': 24464, 'mds_3': 24457, 'mds_4': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/1603518036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 1603518036}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 1603518036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/426074448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 426074448}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 426074448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6833/4042514104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 4042514104}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 4042514104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/2095779466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 2095779466}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 2095779466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/1110459707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 1110459707}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 1110459707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.139:6835/3204357322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 3204357322}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 3204357322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.167:6837/3468047781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 3468047781}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3468047781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.139:6837/2336570561', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 2336570561}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 2336570561}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.167:6839/1858062899', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 1858062899}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 1858062899}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.139:6839/3082363979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 3082363979}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 3082363979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24497, 24466, 24464, 24457, 24449]} 2024-09-05T02:02:29.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-05T02:02:29.757 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.e status 2024-09-05T02:02:30.002 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:30.382 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:30 smithi103 ceph-mon[30902]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-05T02:02:30.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.462+0000 7f4d3a28b640 1 -- 172.21.15.103:0/915592158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 msgr2=0x7f4d34101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:30.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.462+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/915592158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 0x7f4d34101680 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009900 tx=0x7f4d2802a410 comp rx=0 tx=0).stop 2024-09-05T02:02:30.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.462+0000 7f4d3a28b640 1 -- 172.21.15.103:0/915592158 shutdown_connections 2024-09-05T02:02:30.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.462+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/915592158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d34101bc0 0x7f4d3410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.462+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/915592158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 0x7f4d34101680 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.462+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/915592158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34105a90 0x7f4d34105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.462+0000 7f4d3a28b640 1 -- 172.21.15.103:0/915592158 >> 172.21.15.103:0/915592158 conn(0x7f4d34077210 msgr2=0x7f4d340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.462+0000 7f4d3a28b640 1 -- 172.21.15.103:0/915592158 shutdown_connections 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.462+0000 7f4d3a28b640 1 -- 172.21.15.103:0/915592158 wait complete. 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d3a28b640 1 Processor -- start 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d3a28b640 1 -- start start 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d3a28b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 0x7f4d34198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d3a28b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34101bc0 0x7f4d34198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d3a28b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d34105a90 0x7f4d3419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d3a28b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34110e60 con 0x7f4d34101240 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d3a28b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34110ce0 con 0x7f4d34105a90 2024-09-05T02:02:30.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d3a28b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34110fe0 con 0x7f4d34101bc0 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 0x7f4d34198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 0x7f4d34198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58606/0 (socket says 172.21.15.103:58606) 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.463+0000 7f4d33fff640 1 -- 172.21.15.103:0/2123394353 learned_addr learned my addr 172.21.15.103:0/2123394353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d33fff640 1 -- 172.21.15.103:0/2123394353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34101bc0 msgr2=0x7f4d34198950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34101bc0 0x7f4d34198950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d33fff640 1 -- 172.21.15.103:0/2123394353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d34105a90 msgr2=0x7f4d3419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d38828640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d34105a90 0x7f4d3419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d34105a90 0x7f4d3419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d33fff640 1 -- 172.21.15.103:0/2123394353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3419d3c0 con 0x7f4d34101240 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d38828640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d34105a90 0x7f4d3419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 0x7f4d34198410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400c920 tx=0x7f4d2400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d317fa640 1 -- 172.21.15.103:0/2123394353 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24007c70 con 0x7f4d34101240 2024-09-05T02:02:30.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d3a28b640 1 -- 172.21.15.103:0/2123394353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3419d6b0 con 0x7f4d34101240 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d317fa640 1 -- 172.21.15.103:0/2123394353 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d24007e10 con 0x7f4d34101240 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d317fa640 1 -- 172.21.15.103:0/2123394353 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24018820 con 0x7f4d34101240 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.464+0000 7f4d3a28b640 1 -- 172.21.15.103:0/2123394353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d341a4f90 con 0x7f4d34101240 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.466+0000 7f4d317fa640 1 -- 172.21.15.103:0/2123394353 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4d240189c0 con 0x7f4d34101240 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.466+0000 7f4d3c664740 1 Processor -- start 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.466+0000 7f4d3c664740 1 -- start start 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13e9d0 0x5636cd13edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 0x5636cd149670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd149bb0 0x5636cd14bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d3c664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5636cd0a5090 con 0x5636cd149bb0 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d3c664740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5636cd04ca70 con 0x5636cd13e9d0 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d3c664740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636cd0a1050 con 0x5636cd13f2f0 2024-09-05T02:02:30.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd149bb0 0x5636cd14bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd149bb0 0x5636cd14bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58612/0 (socket says 172.21.15.103:58612) 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d337fe640 1 -- 172.21.15.103:0/1529122 learned_addr learned my addr 172.21.15.103:0/1529122 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d33fff640 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 0x5636cd149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d317fa640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d1c0a2b80 0x7f4d1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d38828640 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13e9d0 0x5636cd13edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d337fe640 1 -- 172.21.15.103:0/1529122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 msgr2=0x5636cd149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d337fe640 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 0x5636cd149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d337fe640 1 -- 172.21.15.103:0/1529122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13e9d0 msgr2=0x5636cd13edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d337fe640 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13e9d0 0x5636cd13edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d337fe640 1 -- 172.21.15.103:0/1529122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636cd059130 con 0x5636cd149bb0 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d38828640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d1c0a2b80 0x7f4d1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d317fa640 1 -- 172.21.15.103:0/2123394353 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4d240cf5f0 con 0x7f4d34101240 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d33fff640 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 0x5636cd149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:30.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.467+0000 7f4d337fe640 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd149bb0 0x5636cd14bfa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4d28005740 tx=0x7f4d28032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2003687effad39f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d1a7fc640 1 -- 172.21.15.103:0/1529122 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28040070 con 0x5636cd149bb0 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d38828640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d1c0a2b80 0x7f4d1c0a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c007fb0 tx=0x7f4d2c004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d1a7fc640 1 -- 172.21.15.103:0/1529122 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d28004040 con 0x5636cd149bb0 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d1a7fc640 1 -- 172.21.15.103:0/1529122 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28035040 con 0x5636cd149bb0 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d3c664740 1 -- 172.21.15.103:0/1529122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd149bb0 msgr2=0x5636cd14bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d3c664740 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd149bb0 0x5636cd14bfa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4d28005740 tx=0x7f4d28032c00 comp rx=0 tx=0).stop 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d3c664740 1 -- 172.21.15.103:0/1529122 shutdown_connections 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d3c664740 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd149bb0 0x5636cd14bfa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d3c664740 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 0x5636cd149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d3c664740 1 --2- 172.21.15.103:0/1529122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13e9d0 0x5636cd13edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.468+0000 7f4d3c664740 1 -- 172.21.15.103:0/1529122 >> 172.21.15.103:0/1529122 conn(0x5636cd13af60 msgr2=0x5636cd13b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:30.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 -- 172.21.15.103:0/1529122 shutdown_connections 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 -- 172.21.15.103:0/1529122 wait complete. 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 -- start start 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 Processor -- start 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd149330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 0x5636cd146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 0x5636cd146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5636cd14c780 con 0x5636cd13e9d0 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5636cd05ff20 con 0x5636cd149bb0 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636cd14c900 con 0x5636cd13f2f0 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d3c664740 10 client.0 _ll_register_callbacks cb 0x5636cd029030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd149330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58620/0 (socket says 172.21.15.103:58620) 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.469+0000 7f4d33fff640 1 -- 172.21.15.103:0/2502746790 learned_addr learned my addr 172.21.15.103:0/2502746790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d38828640 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 0x5636cd146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d337fe640 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 0x5636cd146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d33fff640 1 -- 172.21.15.103:0/2502746790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 msgr2=0x5636cd146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 0x5636cd146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d33fff640 1 -- 172.21.15.103:0/2502746790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 msgr2=0x5636cd146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 0x5636cd146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d33fff640 1 -- 172.21.15.103:0/2502746790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636cce162b0 con 0x5636cd13e9d0 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d38828640 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 0x5636cd146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd149330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4d24007590 tx=0x7f4d2409b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156c76d2322867fa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d137fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d137fe640 1 -- 172.21.15.103:0/2502746790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400fb00 con 0x5636cd13e9d0 2024-09-05T02:02:30.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d137fe640 1 -- 172.21.15.103:0/2502746790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4d240cf320 con 0x5636cd13e9d0 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d3c664740 1 -- 172.21.15.103:0/2502746790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d28004040 con 0x5636cd13e9d0 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d137fe640 1 -- 172.21.15.103:0/2502746790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d2400de20 con 0x5636cd13e9d0 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d3c664740 10 client.15231 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d3c664740 1 -- 172.21.15.103:0/2502746790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636cd157fe0 con 0x5636cd13e9d0 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d137fe640 1 -- 172.21.15.103:0/2502746790 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d28004040 con 0x5636cd13e9d0 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d137fe640 1 -- 172.21.15.103:0/2502746790 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4d240ba2a0 con 0x5636cd13e9d0 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.470+0000 7f4d3c664740 10 client.15231 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.471+0000 7f4d3c664740 10 client.15231 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.471+0000 7f4d3c664740 20 client.15231 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.471+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x5636cd15a640 0x5636cd15bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.471+0000 7f4d3c664740 4 client.15231 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.471+0000 7f4d3c664740 1 -- 172.21.15.103:0/2502746790 --> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636cd0b2600 con 0x5636cd15a640 2024-09-05T02:02:30.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.471+0000 7f4d337fe640 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x5636cd15a640 0x5636cd15bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.471+0000 7f4d137fe640 10 client.15231 ms_handle_connect on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.471+0000 7f4d337fe640 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x5636cd15a640 0x5636cd15bab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40646feb024ee394 server_cookie=82061132b0902103 in_seq=0 out_seq=0 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d137fe640 1 -- 172.21.15.103:0/2502746790 <== mds.4 v2:172.21.15.139:6834/3204357322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636cd0b2600 con 0x5636cd15a640 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d137fe640 10 client.15231 handle_command_reply: tid=0 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 client.15231 shutdown 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 -- 172.21.15.103:0/2502746790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 msgr2=0x5636cd149330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd149330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4d24007590 tx=0x7f4d2409b930 comp rx=0 tx=0).stop 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 -- 172.21.15.103:0/2502746790 shutdown_connections 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x5636cd15a640 0x5636cd15bab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 0x5636cd146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13f2f0 0x5636cd146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d137fe640 0 client.15231 ms_handle_reset on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2502746790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd149330 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 -- 172.21.15.103:0/2502746790 >> 172.21.15.103:0/2502746790 conn(0x5636cd13af60 msgr2=0x5636cd140c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:30.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 -- 172.21.15.103:0/2502746790 shutdown_connections 2024-09-05T02:02:30.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 1 -- 172.21.15.103:0/2502746790 wait complete. 2024-09-05T02:02:30.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.472+0000 7f4d3c664740 20 client.15231 trim_cache size 0 max 16384 2024-09-05T02:02:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d3c664740 1 Processor -- start 2024-09-05T02:02:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d3c664740 1 -- start start 2024-09-05T02:02:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13f2f0 0x5636cd146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd149bb0 0x5636cd146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d3c664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5636cd159df0 con 0x5636cd13e9d0 2024-09-05T02:02:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d3c664740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5636cd15a4c0 con 0x5636cd13f2f0 2024-09-05T02:02:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d3c664740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636cd15ab90 con 0x5636cd149bb0 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd145f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58636/0 (socket says 172.21.15.103:58636) 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d33fff640 1 -- 172.21.15.103:0/2403731111 learned_addr learned my addr 172.21.15.103:0/2403731111 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d337fe640 1 --2- 172.21.15.103:0/2403731111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13f2f0 0x5636cd146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d33fff640 1 -- 172.21.15.103:0/2403731111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd149bb0 msgr2=0x5636cd146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2403731111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd149bb0 0x5636cd146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d33fff640 1 -- 172.21.15.103:0/2403731111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13f2f0 msgr2=0x5636cd146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2403731111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13f2f0 0x5636cd146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d33fff640 1 -- 172.21.15.103:0/2403731111 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636cd1496a0 con 0x5636cd13e9d0 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.490+0000 7f4d337fe640 1 --2- 172.21.15.103:0/2403731111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13f2f0 0x5636cd146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2403731111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd145f40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4d240b6040 tx=0x7f4d2400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1585e7cbbebc23 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d19ffb640 1 -- 172.21.15.103:0/2403731111 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400fb00 con 0x5636cd13e9d0 2024-09-05T02:02:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d19ffb640 1 -- 172.21.15.103:0/2403731111 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d2409c1a0 con 0x5636cd13e9d0 2024-09-05T02:02:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d19ffb640 1 -- 172.21.15.103:0/2403731111 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d240977f0 con 0x5636cd13e9d0 2024-09-05T02:02:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d3c664740 1 -- 172.21.15.103:0/2403731111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 msgr2=0x5636cd145f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2403731111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd145f40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4d240b6040 tx=0x7f4d2400cf90 comp rx=0 tx=0).stop 2024-09-05T02:02:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d3c664740 1 -- 172.21.15.103:0/2403731111 shutdown_connections 2024-09-05T02:02:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2403731111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd149bb0 0x5636cd146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2403731111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd13f2f0 0x5636cd146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2403731111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13e9d0 0x5636cd145f40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.491+0000 7f4d3c664740 1 -- 172.21.15.103:0/2403731111 >> 172.21.15.103:0/2403731111 conn(0x5636cd13af60 msgr2=0x5636cd135ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 -- 172.21.15.103:0/2403731111 shutdown_connections 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 -- 172.21.15.103:0/2403731111 wait complete. 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 -- start start 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 Processor -- start 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13e9d0 0x5636cd15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13f2f0 0x5636cd15f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 0x5636cd15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5636cd15b930 con 0x5636cd13f2f0 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5636cd15b260 con 0x5636cd149bb0 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636cd15c000 con 0x5636cd13e9d0 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d3c664740 10 client.0 _ll_register_callbacks cb 0x7f4d2400a5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13f2f0 0x5636cd15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13f2f0 0x5636cd15f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58646/0 (socket says 172.21.15.103:58646) 2024-09-05T02:02:30.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.492+0000 7f4d337fe640 1 -- 172.21.15.103:0/2761066195 learned_addr learned my addr 172.21.15.103:0/2761066195 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d337fe640 1 -- 172.21.15.103:0/2761066195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13e9d0 msgr2=0x5636cd15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d337fe640 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13e9d0 0x5636cd15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d337fe640 1 -- 172.21.15.103:0/2761066195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 msgr2=0x5636cd15f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d38828640 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 0x5636cd15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d337fe640 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 0x5636cd15f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d337fe640 1 -- 172.21.15.103:0/2761066195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636cd1c2f60 con 0x5636cd13f2f0 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d337fe640 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13f2f0 0x5636cd15f270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4d28006010 tx=0x7f4d28005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6eae67593e145f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d137fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d137fe640 1 -- 172.21.15.103:0/2761066195 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28035040 con 0x5636cd13f2f0 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d137fe640 1 -- 172.21.15.103:0/2761066195 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4d28032450 con 0x5636cd13f2f0 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d3c664740 1 -- 172.21.15.103:0/2761066195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d2409c1a0 con 0x5636cd13f2f0 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d137fe640 1 -- 172.21.15.103:0/2761066195 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d28040040 con 0x5636cd13f2f0 2024-09-05T02:02:30.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d3c664740 10 client.15237 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d3c664740 1 -- 172.21.15.103:0/2761066195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636cd1c4240 con 0x5636cd13f2f0 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.493+0000 7f4d137fe640 1 -- 172.21.15.103:0/2761066195 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d2409c1a0 con 0x5636cd13f2f0 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d137fe640 1 -- 172.21.15.103:0/2761066195 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4d28045a20 con 0x5636cd13f2f0 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d3c664740 10 client.15237 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d3c664740 10 client.15237 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d3c664740 20 client.15237 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x5636cd1c89e0 0x5636cd1c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d3c664740 4 client.15237 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d3c664740 1 -- 172.21.15.103:0/2761066195 --> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] -- command(tid 0: {"prefix": "status"}) -- 0x5636cd0acb10 con 0x5636cd1c89e0 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x5636cd1c89e0 0x5636cd1c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d137fe640 10 client.15237 ms_handle_connect on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.494+0000 7f4d33fff640 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x5636cd1c89e0 0x5636cd1c76f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c04d22ed0f2eb540 server_cookie=ba8dffea9704077b in_seq=0 out_seq=0 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d137fe640 1 -- 172.21.15.103:0/2761066195 <== mds.4 v2:172.21.15.139:6834/3204357322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5636cd0acb10 con 0x5636cd1c89e0 2024-09-05T02:02:30.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d137fe640 10 client.15237 handle_command_reply: tid=0 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 client.15237 shutdown 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 -- 172.21.15.103:0/2761066195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13f2f0 msgr2=0x5636cd15f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13f2f0 0x5636cd15f270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4d28006010 tx=0x7f4d28005d50 comp rx=0 tx=0).stop 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 -- 172.21.15.103:0/2761066195 shutdown_connections 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x5636cd1c89e0 0x5636cd1c76f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5636cd149bb0 0x5636cd15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636cd13f2f0 0x5636cd15f270 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d137fe640 0 client.15237 ms_handle_reset on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 --2- 172.21.15.103:0/2761066195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636cd13e9d0 0x5636cd15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 -- 172.21.15.103:0/2761066195 >> 172.21.15.103:0/2761066195 conn(0x5636cd13af60 msgr2=0x5636cd146640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 -- 172.21.15.103:0/2761066195 shutdown_connections 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 1 -- 172.21.15.103:0/2761066195 wait complete. 2024-09-05T02:02:30.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3c664740 20 client.15237 trim_cache size 0 max 16384 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24449, 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 92.27419209, 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:02:30.505 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:30.506 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 168.00501009600001 2024-09-05T02:02:30.506 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:30.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3a28b640 1 -- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d1c0a2b80 msgr2=0x7f4d1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d1c0a2b80 0x7f4d1c0a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c007fb0 tx=0x7f4d2c004b50 comp rx=0 tx=0).stop 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3a28b640 1 -- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 msgr2=0x7f4d34198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.495+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 0x7f4d34198410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400c920 tx=0x7f4d2400cde0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.496+0000 7f4d3a28b640 1 -- 172.21.15.103:0/2123394353 shutdown_connections 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.496+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d34105a90 0x7f4d3419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.496+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d1c0a2b80 0x7f4d1c0a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.496+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34101bc0 0x7f4d34198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.496+0000 7f4d3a28b640 1 --2- 172.21.15.103:0/2123394353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d34101240 0x7f4d34198410 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.496+0000 7f4d3a28b640 1 -- 172.21.15.103:0/2123394353 >> 172.21.15.103:0/2123394353 conn(0x7f4d34077210 msgr2=0x7f4d34100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.496+0000 7f4d3a28b640 1 -- 172.21.15.103:0/2123394353 shutdown_connections 2024-09-05T02:02:30.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:30.496+0000 7f4d3a28b640 1 -- 172.21.15.103:0/2123394353 wait complete. 2024-09-05T02:02:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:30 smithi167 ceph-mon[35122]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-05T02:02:30.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:30 smithi139 ceph-mon[34262]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-05T02:02:31.219 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.c status 2024-09-05T02:02:31.465 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:31.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.929+0000 7fbf88694640 1 -- 172.21.15.103:0/3113820425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80102790 msgr2=0x7fbf801067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.929+0000 7fbf88694640 1 --2- 172.21.15.103:0/3113820425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80102790 0x7fbf801067e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009900 tx=0x7fbf7402a410 comp rx=0 tx=0).stop 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.929+0000 7fbf88694640 1 -- 172.21.15.103:0/3113820425 shutdown_connections 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.929+0000 7fbf88694640 1 --2- 172.21.15.103:0/3113820425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf80106f10 0x7fbf8010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.929+0000 7fbf88694640 1 --2- 172.21.15.103:0/3113820425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80102790 0x7fbf801067e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.929+0000 7fbf88694640 1 --2- 172.21.15.103:0/3113820425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf80101de0 0x7fbf801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.929+0000 7fbf88694640 1 -- 172.21.15.103:0/3113820425 >> 172.21.15.103:0/3113820425 conn(0x7fbf800fd8c0 msgr2=0x7fbf800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.929+0000 7fbf88694640 1 -- 172.21.15.103:0/3113820425 shutdown_connections 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.929+0000 7fbf88694640 1 -- 172.21.15.103:0/3113820425 wait complete. 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf88694640 1 Processor -- start 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf88694640 1 -- start start 2024-09-05T02:02:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf88694640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80101de0 0x7fbf80198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf88694640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf80102790 0x7fbf80198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf88694640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf80106f10 0x7fbf8019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf88694640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80111ec0 con 0x7fbf80101de0 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf88694640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80111d40 con 0x7fbf80102790 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf88694640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80112040 con 0x7fbf80106f10 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf86430640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80101de0 0x7fbf80198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf86430640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80101de0 0x7fbf80198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58672/0 (socket says 172.21.15.103:58672) 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf86430640 1 -- 172.21.15.103:0/2101075780 learned_addr learned my addr 172.21.15.103:0/2101075780 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.930+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf80102790 0x7fbf80198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf86430640 1 -- 172.21.15.103:0/2101075780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf80106f10 msgr2=0x7fbf8019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf86430640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf80106f10 0x7fbf8019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf86430640 1 -- 172.21.15.103:0/2101075780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf80102790 msgr2=0x7fbf80198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf86430640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf80102790 0x7fbf80198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf86430640 1 -- 172.21.15.103:0/2101075780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8019d4d0 con 0x7fbf80101de0 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf80102790 0x7fbf80198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf86430640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80101de0 0x7fbf80198520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000ba50 tx=0x7fbf7000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf7f7fe640 1 -- 172.21.15.103:0/2101075780 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70019070 con 0x7fbf80101de0 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf7f7fe640 1 -- 172.21.15.103:0/2101075780 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf7000cbe0 con 0x7fbf80101de0 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf7f7fe640 1 -- 172.21.15.103:0/2101075780 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700075d0 con 0x7fbf80101de0 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf88694640 1 -- 172.21.15.103:0/2101075780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8019d7c0 con 0x7fbf80101de0 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.931+0000 7fbf88694640 1 -- 172.21.15.103:0/2101075780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf801a50a0 con 0x7fbf80101de0 2024-09-05T02:02:31.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.933+0000 7fbf7f7fe640 1 -- 172.21.15.103:0/2101075780 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbf7001c070 con 0x7fbf80101de0 2024-09-05T02:02:31.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.933+0000 7fbf8aa6d740 1 Processor -- start 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf8aa6d740 1 -- start start 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf7f7fe640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbf680a2c00 0x7fbf680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 0x564a25e55c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e561d0 0x564a25e64a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e64fc0 0x564a25e673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a25dbc090 con 0x564a25e561d0 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564a25d63a70 con 0x564a25e64fc0 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564a25db8050 con 0x564a25e558b0 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf86c31640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbf680a2c00 0x7fbf680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf85c2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e561d0 0x564a25e64a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf85c2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e561d0 0x564a25e64a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58686/0 (socket says 172.21.15.103:58686) 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf85c2f640 1 -- 172.21.15.103:0/3080235177 learned_addr learned my addr 172.21.15.103:0/3080235177 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf7f7fe640 1 -- 172.21.15.103:0/2101075780 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbf700cea40 con 0x7fbf80101de0 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf86430640 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 0x564a25e55c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.934+0000 7fbf86c31640 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e64fc0 0x564a25e673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf85c2f640 1 -- 172.21.15.103:0/3080235177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 msgr2=0x564a25e55c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 0x564a25e55c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf85c2f640 1 -- 172.21.15.103:0/3080235177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e64fc0 msgr2=0x564a25e673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e64fc0 0x564a25e673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf85c2f640 1 -- 172.21.15.103:0/3080235177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a25d70130 con 0x564a25e561d0 2024-09-05T02:02:31.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf86430640 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 0x564a25e55c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf86c31640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbf680a2c00 0x7fbf680a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009c60 tx=0x7fbf780092d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e561d0 0x564a25e64a80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf7402a930 tx=0x7fbf74031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5021cc9f757c1d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf7cff9640 1 -- 172.21.15.103:0/3080235177 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7403f070 con 0x564a25e561d0 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf7cff9640 1 -- 172.21.15.103:0/3080235177 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf740316a0 con 0x564a25e561d0 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.935+0000 7fbf7cff9640 1 -- 172.21.15.103:0/3080235177 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74035070 con 0x564a25e561d0 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3080235177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e561d0 msgr2=0x564a25e64a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e561d0 0x564a25e64a80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf7402a930 tx=0x7fbf74031e40 comp rx=0 tx=0).stop 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3080235177 shutdown_connections 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e64fc0 0x564a25e673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e561d0 0x564a25e64a80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3080235177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 0x564a25e55c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3080235177 >> 172.21.15.103:0/3080235177 conn(0x564a25e51f60 msgr2=0x564a25e52340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3080235177 shutdown_connections 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3080235177 wait complete. 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 -- start start 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.936+0000 7fbf8aa6d740 1 Processor -- start 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e64820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e61b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 0x564a25e620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a25e67b90 con 0x564a25e558b0 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564a25d76f20 con 0x564a25e561d0 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564a25e67d10 con 0x564a25e64fc0 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf8aa6d740 10 client.0 _ll_register_callbacks cb 0x564a25d40030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86430640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e64820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86430640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e64820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58700/0 (socket says 172.21.15.103:58700) 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86430640 1 -- 172.21.15.103:0/3948289391 learned_addr learned my addr 172.21.15.103:0/3948289391 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:31.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e61b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86c31640 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 0x564a25e620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86430640 1 -- 172.21.15.103:0/3948289391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 msgr2=0x564a25e620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86430640 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 0x564a25e620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86430640 1 -- 172.21.15.103:0/3948289391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 msgr2=0x564a25e61b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86430640 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e61b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86430640 1 -- 172.21.15.103:0/3948289391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a25b2d2b0 con 0x564a25e558b0 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e61b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf86430640 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e64820 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000ba20 tx=0x7fbf7009b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52a15e567618bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf65ffb640 1 -- 172.21.15.103:0/3948289391 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7009c220 con 0x564a25e558b0 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf65ffb640 1 -- 172.21.15.103:0/3948289391 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbf700aad60 con 0x564a25e558b0 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.937+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3948289391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf740316a0 con 0x564a25e558b0 2024-09-05T02:02:31.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf65ffb640 1 -- 172.21.15.103:0/3948289391 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf700aaf80 con 0x564a25e558b0 2024-09-05T02:02:31.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf8aa6d740 10 client.15249 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:31.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3948289391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a25e73520 con 0x564a25e558b0 2024-09-05T02:02:31.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf65ffb640 1 -- 172.21.15.103:0/3948289391 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf740316a0 con 0x564a25e558b0 2024-09-05T02:02:31.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf65ffb640 1 -- 172.21.15.103:0/3948289391 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbf700a36d0 con 0x564a25e558b0 2024-09-05T02:02:31.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf8aa6d740 10 client.15249 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:31.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf8aa6d740 10 client.15249 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T02:02:31.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf8aa6d740 20 client.15249 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:31.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x564a25e75ad0 0x564a25e76f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf8aa6d740 4 client.15249 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3948289391 --> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a25dc9600 con 0x564a25e75ad0 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.938+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x564a25e75ad0 0x564a25e76f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.939+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x564a25e75ad0 0x564a25e76f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3621cce5719f65aa server_cookie=3d25f59d19a95a0d in_seq=0 out_seq=0 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.939+0000 7fbf65ffb640 10 client.15249 ms_handle_connect on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.939+0000 7fbf65ffb640 1 -- 172.21.15.103:0/3948289391 <== mds.3 v2:172.21.15.167:6836/3468047781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a25dc9600 con 0x564a25e75ad0 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.939+0000 7fbf65ffb640 10 client.15249 handle_command_reply: tid=0 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.939+0000 7fbf8aa6d740 1 client.15249 shutdown 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3948289391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 msgr2=0x564a25e64820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e64820 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000ba20 tx=0x7fbf7009b5f0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3948289391 shutdown_connections 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x564a25e75ad0 0x564a25e76f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 0x564a25e620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e61b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/3948289391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e64820 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3948289391 >> 172.21.15.103:0/3948289391 conn(0x564a25e51f60 msgr2=0x564a25e57cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:31.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf65ffb640 0 client.15249 ms_handle_reset on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:31.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3948289391 shutdown_connections 2024-09-05T02:02:31.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/3948289391 wait complete. 2024-09-05T02:02:31.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.940+0000 7fbf8aa6d740 20 client.15249 trim_cache size 0 max 16384 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.957+0000 7fbf8aa6d740 1 Processor -- start 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.957+0000 7fbf8aa6d740 1 -- start start 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.957+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 0x564a25e75ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e60440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e64fc0 0x564a25e60980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a25e75280 con 0x564a25e64fc0 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564a25e75950 con 0x564a25e561d0 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564a25e76020 con 0x564a25e558b0 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86c31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e64fc0 0x564a25e60980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86c31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e64fc0 0x564a25e60980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58710/0 (socket says 172.21.15.103:58710) 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86c31640 1 -- 172.21.15.103:0/2445920660 learned_addr learned my addr 172.21.15.103:0/2445920660 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86c31640 1 -- 172.21.15.103:0/2445920660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 msgr2=0x564a25e75ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/2445920660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e60440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86430640 1 --2- 172.21.15.103:0/2445920660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 0x564a25e75ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86c31640 1 --2- 172.21.15.103:0/2445920660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 0x564a25e75ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86c31640 1 -- 172.21.15.103:0/2445920660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 msgr2=0x564a25e60440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86c31640 1 --2- 172.21.15.103:0/2445920660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e60440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86c31640 1 -- 172.21.15.103:0/2445920660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a25e64b90 con 0x564a25e64fc0 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf86c31640 1 --2- 172.21.15.103:0/2445920660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e64fc0 0x564a25e60980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009100 tx=0x7fbf78018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a463216c3ecfeb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf67fff640 1 -- 172.21.15.103:0/2445920660 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf780035f0 con 0x564a25e64fc0 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf67fff640 1 -- 172.21.15.103:0/2445920660 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf78010040 con 0x564a25e64fc0 2024-09-05T02:02:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.958+0000 7fbf67fff640 1 -- 172.21.15.103:0/2445920660 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78023a70 con 0x564a25e64fc0 2024-09-05T02:02:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/2445920660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e64fc0 msgr2=0x564a25e60980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/2445920660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e64fc0 0x564a25e60980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009100 tx=0x7fbf78018000 comp rx=0 tx=0).stop 2024-09-05T02:02:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/2445920660 shutdown_connections 2024-09-05T02:02:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/2445920660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e64fc0 0x564a25e60980 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/2445920660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e60440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/2445920660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e558b0 0x564a25e75ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/2445920660 >> 172.21.15.103:0/2445920660 conn(0x564a25e51f60 msgr2=0x564a25e61b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/2445920660 shutdown_connections 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/2445920660 wait complete. 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 -- start start 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.959+0000 7fbf8aa6d740 1 Processor -- start 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e5e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e5eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf8aa6d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 0x564a25e7e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a25e76dc0 con 0x564a25e558b0 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564a25e766f0 con 0x564a25e561d0 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf8aa6d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564a25e77490 con 0x564a25e64fc0 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf8aa6d740 10 client.0 _ll_register_callbacks cb 0x7fbf7009a850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86430640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e5e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86430640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e5e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58722/0 (socket says 172.21.15.103:58722) 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86430640 1 -- 172.21.15.103:0/1684301085 learned_addr learned my addr 172.21.15.103:0/1684301085 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86c31640 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 0x564a25e7e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86430640 1 -- 172.21.15.103:0/1684301085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 msgr2=0x564a25e7e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86430640 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 0x564a25e7e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86430640 1 -- 172.21.15.103:0/1684301085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 msgr2=0x564a25e5eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86430640 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e5eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86430640 1 -- 172.21.15.103:0/1684301085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a25e7e610 con 0x564a25e558b0 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86c31640 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 0x564a25e7e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf86430640 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e5e560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbf700a3630 tx=0x7fbf7009ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4adddd072c73a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf65ffb640 1 -- 172.21.15.103:0/1684301085 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70097190 con 0x564a25e558b0 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf65ffb640 1 -- 172.21.15.103:0/1684301085 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbf700a36d0 con 0x564a25e558b0 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/1684301085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf78010040 con 0x564a25e558b0 2024-09-05T02:02:31.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf65ffb640 1 -- 172.21.15.103:0/1684301085 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf7009f2a0 con 0x564a25e558b0 2024-09-05T02:02:31.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.960+0000 7fbf8aa6d740 10 client.15255 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:31.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/1684301085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a25e7f8b0 con 0x564a25e558b0 2024-09-05T02:02:31.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf65ffb640 1 -- 172.21.15.103:0/1684301085 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf78010040 con 0x564a25e558b0 2024-09-05T02:02:31.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf65ffb640 1 -- 172.21.15.103:0/1684301085 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbf700aa050 con 0x564a25e558b0 2024-09-05T02:02:31.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf8aa6d740 10 client.15255 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:31.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf8aa6d740 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T02:02:31.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf8aa6d740 20 client.15255 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:31.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x564a25ee4470 0x564a25ee3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:31.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf8aa6d740 4 client.15255 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:31.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/1684301085 --> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] -- command(tid 0: {"prefix": "status"}) -- 0x564a25dc3b10 con 0x564a25ee4470 2024-09-05T02:02:31.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.961+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x564a25ee4470 0x564a25ee3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:31.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf65ffb640 10 client.15255 ms_handle_connect on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:31.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf85c2f640 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x564a25ee4470 0x564a25ee3180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69fe1478a7ed494c server_cookie=c8b5686358de33fb in_seq=0 out_seq=0 2024-09-05T02:02:31.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf65ffb640 1 -- 172.21.15.103:0/1684301085 <== mds.3 v2:172.21.15.167:6836/3468047781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x564a25dc3b10 con 0x564a25ee4470 2024-09-05T02:02:31.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf65ffb640 10 client.15255 handle_command_reply: tid=0 2024-09-05T02:02:31.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 client.15255 shutdown 2024-09-05T02:02:31.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/1684301085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 msgr2=0x564a25e5e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:31.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e5e560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbf700a3630 tx=0x7fbf7009ade0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/1684301085 shutdown_connections 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x564a25ee4470 0x564a25ee3180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf65ffb640 0 client.15255 ms_handle_reset on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a25e64fc0 0x564a25e7e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a25e561d0 0x564a25e5eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 --2- 172.21.15.103:0/1684301085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a25e558b0 0x564a25e5e560 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/1684301085 >> 172.21.15.103:0/1684301085 conn(0x564a25e51f60 msgr2=0x564a25e603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/1684301085 shutdown_connections 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 1 -- 172.21.15.103:0/1684301085 wait complete. 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.962+0000 7fbf8aa6d740 20 client.15255 trim_cache size 0 max 16384 2024-09-05T02:02:31.969 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24457, 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 94.754373719, 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 163.71879983700001 2024-09-05T02:02:31.970 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:31.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 -- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbf680a2c00 msgr2=0x7fbf680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:31.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbf680a2c00 0x7fbf680a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009c60 tx=0x7fbf780092d0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 -- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80101de0 msgr2=0x7fbf80198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:31.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80101de0 0x7fbf80198520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000ba50 tx=0x7fbf7000bf10 comp rx=0 tx=0).stop 2024-09-05T02:02:31.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 -- 172.21.15.103:0/2101075780 shutdown_connections 2024-09-05T02:02:31.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbf680a2c00 0x7fbf680a50c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf80106f10 0x7fbf8019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf80102790 0x7fbf80198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 --2- 172.21.15.103:0/2101075780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf80101de0 0x7fbf80198520 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:31.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 -- 172.21.15.103:0/2101075780 >> 172.21.15.103:0/2101075780 conn(0x7fbf800fd8c0 msgr2=0x7fbf800fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:31.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 -- 172.21.15.103:0/2101075780 shutdown_connections 2024-09-05T02:02:31.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:31.963+0000 7fbf88694640 1 -- 172.21.15.103:0/2101075780 wait complete. 2024-09-05T02:02:32.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:32 smithi103 ceph-mon[30902]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 895 B/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:32 smithi167 ceph-mon[35122]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 895 B/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:32.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:32 smithi139 ceph-mon[34262]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 895 B/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:32.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T02:02:33.021 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:33.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4163100350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54100e20 msgr2=0x7f0f54101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:33.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4163100350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54100e20 0x7f0f54101200 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009900 tx=0x7f0f5002a480 comp rx=0 tx=0).stop 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4163100350 shutdown_connections 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4163100350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f54102ef0 0x7f0f5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4163100350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f54102570 0x7f0f541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4163100350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54100e20 0x7f0f54101200 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4163100350 >> 172.21.15.103:0/4163100350 conn(0x7f0f540fc7d0 msgr2=0x7f0f540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4163100350 shutdown_connections 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4163100350 wait complete. 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.532+0000 7f0f5c06b640 1 Processor -- start 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5c06b640 1 -- start start 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5c06b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f54100e20 0x7f0f54196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5c06b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f54102570 0x7f0f54197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5c06b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54102ef0 0x7f0f54197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5c06b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54110e30 con 0x7f0f54102ef0 2024-09-05T02:02:33.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5c06b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54110cb0 con 0x7f0f54102570 2024-09-05T02:02:33.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5c06b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54110fb0 con 0x7f0f54100e20 2024-09-05T02:02:33.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5a608640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54102ef0 0x7f0f54197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5a608640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54102ef0 0x7f0f54197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58736/0 (socket says 172.21.15.103:58736) 2024-09-05T02:02:33.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5a608640 1 -- 172.21.15.103:0/4204613566 learned_addr learned my addr 172.21.15.103:0/4204613566 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:33.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f59e07640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f54100e20 0x7f0f54196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5a608640 1 -- 172.21.15.103:0/4204613566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f54100e20 msgr2=0x7f0f54196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:33.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5a608640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f54100e20 0x7f0f54196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f59606640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f54102570 0x7f0f54197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5a608640 1 -- 172.21.15.103:0/4204613566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f54102570 msgr2=0x7f0f54197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5a608640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f54102570 0x7f0f54197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5a608640 1 -- 172.21.15.103:0/4204613566 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f541a6860 con 0x7f0f54102ef0 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f59e07640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f54100e20 0x7f0f54196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f59606640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f54102570 0x7f0f54197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.533+0000 7f0f5a608640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54102ef0 0x7f0f54197a70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00cce0 tx=0x7f0f4c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.534+0000 7f0f46ffd640 1 -- 172.21.15.103:0/4204613566 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00f450 con 0x7f0f54102ef0 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.534+0000 7f0f46ffd640 1 -- 172.21.15.103:0/4204613566 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f4c00faa0 con 0x7f0f54102ef0 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.534+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4204613566 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f541a6b50 con 0x7f0f54102ef0 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.534+0000 7f0f46ffd640 1 -- 172.21.15.103:0/4204613566 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c018b70 con 0x7f0f54102ef0 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.534+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4204613566 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f541a7040 con 0x7f0f54102ef0 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.536+0000 7f0f46ffd640 1 -- 172.21.15.103:0/4204613566 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0f4c018d10 con 0x7f0f54102ef0 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.536+0000 7f0f5e444740 1 Processor -- start 2024-09-05T02:02:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.536+0000 7f0f5e444740 1 -- start start 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b41db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b422f0 0x556772b4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b4cbb0 0x556772b4efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f5e444740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556772aa8090 con 0x556772b419d0 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f5e444740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556772a4fa70 con 0x556772b422f0 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f46ffd640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0f400a2b80 0x7f0f400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f5e444740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556772aa4050 con 0x556772b4cbb0 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59606640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b41db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59606640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b41db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58746/0 (socket says 172.21.15.103:58746) 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59606640 1 -- 172.21.15.103:0/2915236416 learned_addr learned my addr 172.21.15.103:0/2915236416 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f46ffd640 1 -- 172.21.15.103:0/4204613566 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0f4c0cf8e0 con 0x7f0f54102ef0 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f5a608640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0f400a2b80 0x7f0f400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59e07640 1 --2- 172.21.15.103:0/2915236416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b422f0 0x556772b4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59606640 1 -- 172.21.15.103:0/2915236416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b4cbb0 msgr2=0x556772b4efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59606640 1 --2- 172.21.15.103:0/2915236416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b4cbb0 0x556772b4efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59606640 1 -- 172.21.15.103:0/2915236416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b422f0 msgr2=0x556772b4c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59606640 1 --2- 172.21.15.103:0/2915236416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b422f0 0x556772b4c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59606640 1 -- 172.21.15.103:0/2915236416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556772a5c130 con 0x556772b419d0 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f59606640 1 --2- 172.21.15.103:0/2915236416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b41db0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800bbb0 tx=0x7f0f4800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b210a509fd5806e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.537+0000 7f0f5a608640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0f400a2b80 0x7f0f400a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0a44d0 tx=0x7f0f4c020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f3ffff640 1 -- 172.21.15.103:0/2915236416 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48011670 con 0x556772b419d0 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f3ffff640 1 -- 172.21.15.103:0/2915236416 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f48011cc0 con 0x556772b419d0 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f3ffff640 1 -- 172.21.15.103:0/2915236416 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48019b50 con 0x556772b419d0 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f5e444740 1 -- 172.21.15.103:0/2915236416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 msgr2=0x556772b41db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:33.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2915236416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b41db0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800bbb0 tx=0x7f0f4800f730 comp rx=0 tx=0).stop 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f5e444740 1 -- 172.21.15.103:0/2915236416 shutdown_connections 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2915236416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b4cbb0 0x556772b4efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2915236416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b422f0 0x556772b4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2915236416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b41db0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f5e444740 1 -- 172.21.15.103:0/2915236416 >> 172.21.15.103:0/2915236416 conn(0x556772b3df60 msgr2=0x556772b3e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f5e444740 1 -- 172.21.15.103:0/2915236416 shutdown_connections 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.538+0000 7f0f5e444740 1 -- 172.21.15.103:0/2915236416 wait complete. 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.539+0000 7f0f5e444740 1 -- start start 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.539+0000 7f0f5e444740 1 Processor -- start 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.539+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b419d0 0x556772b4c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.539+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.539+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 0x556772b49cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.539+0000 7f0f5e444740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556772b4f780 con 0x556772b422f0 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.539+0000 7f0f5e444740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556772a62f20 con 0x556772b4cbb0 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.539+0000 7f0f5e444740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556772b4f900 con 0x556772b419d0 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.539+0000 7f0f5e444740 10 client.0 _ll_register_callbacks cb 0x556772a2c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59e07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59e07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b497b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58748/0 (socket says 172.21.15.103:58748) 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59e07640 1 -- 172.21.15.103:0/4026882478 learned_addr learned my addr 172.21.15.103:0/4026882478 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59606640 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 0x556772b49cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59606640 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b419d0 0x556772b4c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59e07640 1 -- 172.21.15.103:0/4026882478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b419d0 msgr2=0x556772b4c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59e07640 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b419d0 0x556772b4c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59e07640 1 -- 172.21.15.103:0/4026882478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 msgr2=0x556772b49cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59e07640 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 0x556772b49cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59e07640 1 -- 172.21.15.103:0/4026882478 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567728192b0 con 0x556772b422f0 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f59e07640 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b497b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009900 tx=0x7f0f50031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9e4a4863c65d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/4026882478 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5003f070 con 0x556772b422f0 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/4026882478 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0f50004450 con 0x556772b422f0 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f5e444740 1 -- 172.21.15.103:0/4026882478 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f48011cc0 con 0x556772b422f0 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.540+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/4026882478 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0f5004b050 con 0x556772b422f0 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f5e444740 10 client.15267 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f5e444740 1 -- 172.21.15.103:0/4026882478 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556772b5b0d0 con 0x556772b422f0 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/4026882478 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f48011cc0 con 0x556772b422f0 2024-09-05T02:02:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/4026882478 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f0f50004750 con 0x556772b422f0 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f5e444740 10 client.15267 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f5e444740 10 client.15267 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f5e444740 20 client.15267 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f5e444740 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x556772b5d730 0x556772b5eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f5e444740 4 client.15267 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f5e444740 1 -- 172.21.15.103:0/4026882478 --> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556772ab5600 con 0x556772b5d730 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.541+0000 7f0f59606640 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x556772b5d730 0x556772b5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.542+0000 7f0f3d7fa640 10 client.15267 ms_handle_connect on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.542+0000 7f0f59606640 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x556772b5d730 0x556772b5eba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0e34878efcba09c server_cookie=ba2a6093ffae72ee in_seq=0 out_seq=0 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.542+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/4026882478 <== mds.2 v2:172.21.15.139:6836/2336570561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556772ab5600 con 0x556772b5d730 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.542+0000 7f0f3d7fa640 10 client.15267 handle_command_reply: tid=0 2024-09-05T02:02:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 client.15267 shutdown 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 -- 172.21.15.103:0/4026882478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 msgr2=0x556772b497b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b497b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009900 tx=0x7f0f50031820 comp rx=0 tx=0).stop 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 -- 172.21.15.103:0/4026882478 shutdown_connections 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 0x556772b49cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b497b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x556772b5d730 0x556772b5eba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 --2- 172.21.15.103:0/4026882478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b419d0 0x556772b4c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f3d7fa640 0 client.15267 ms_handle_reset on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 -- 172.21.15.103:0/4026882478 >> 172.21.15.103:0/4026882478 conn(0x556772b3df60 msgr2=0x556772b43c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 -- 172.21.15.103:0/4026882478 shutdown_connections 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 1 -- 172.21.15.103:0/4026882478 wait complete. 2024-09-05T02:02:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.543+0000 7f0f5e444740 20 client.15267 trim_cache size 0 max 16384 2024-09-05T02:02:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f5e444740 1 Processor -- start 2024-09-05T02:02:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f5e444740 1 -- start start 2024-09-05T02:02:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b419d0 0x556772b5eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b46a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b4cbb0 0x556772b46f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f5e444740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556772b5cee0 con 0x556772b422f0 2024-09-05T02:02:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f5e444740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556772b5d5b0 con 0x556772b419d0 2024-09-05T02:02:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f5e444740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556772b5dc80 con 0x556772b4cbb0 2024-09-05T02:02:33.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f59e07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b46a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f59e07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b46a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58760/0 (socket says 172.21.15.103:58760) 2024-09-05T02:02:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f59e07640 1 -- 172.21.15.103:0/351027539 learned_addr learned my addr 172.21.15.103:0/351027539 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f59606640 1 --2- 172.21.15.103:0/351027539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b419d0 0x556772b5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.561+0000 7f0f59606640 1 --2- 172.21.15.103:0/351027539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b4cbb0 0x556772b46f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.562+0000 7f0f59e07640 1 -- 172.21.15.103:0/351027539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b4cbb0 msgr2=0x556772b46f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.562+0000 7f0f59e07640 1 --2- 172.21.15.103:0/351027539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b4cbb0 0x556772b46f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.562+0000 7f0f59e07640 1 -- 172.21.15.103:0/351027539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b419d0 msgr2=0x556772b5eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.562+0000 7f0f59e07640 1 --2- 172.21.15.103:0/351027539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b419d0 0x556772b5eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.562+0000 7f0f59e07640 1 -- 172.21.15.103:0/351027539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556772b4c080 con 0x556772b422f0 2024-09-05T02:02:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.562+0000 7f0f59e07640 1 --2- 172.21.15.103:0/351027539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b46a10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f50004b20 tx=0x7f0f50004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca15e60b9f30530 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.562+0000 7f0f3f7fe640 1 -- 172.21.15.103:0/351027539 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50055070 con 0x556772b422f0 2024-09-05T02:02:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.562+0000 7f0f3f7fe640 1 -- 172.21.15.103:0/351027539 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f50048040 con 0x556772b422f0 2024-09-05T02:02:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.562+0000 7f0f3f7fe640 1 -- 172.21.15.103:0/351027539 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5003f070 con 0x556772b422f0 2024-09-05T02:02:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 -- 172.21.15.103:0/351027539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 msgr2=0x556772b46a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 --2- 172.21.15.103:0/351027539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b46a10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f50004b20 tx=0x7f0f50004890 comp rx=0 tx=0).stop 2024-09-05T02:02:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 -- 172.21.15.103:0/351027539 shutdown_connections 2024-09-05T02:02:33.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 --2- 172.21.15.103:0/351027539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b4cbb0 0x556772b46f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 --2- 172.21.15.103:0/351027539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b422f0 0x556772b46a10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 --2- 172.21.15.103:0/351027539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b419d0 0x556772b5eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 -- 172.21.15.103:0/351027539 >> 172.21.15.103:0/351027539 conn(0x556772b3df60 msgr2=0x556772b48080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 -- 172.21.15.103:0/351027539 shutdown_connections 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 -- 172.21.15.103:0/351027539 wait complete. 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.563+0000 7f0f5e444740 1 -- start start 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f5e444740 1 Processor -- start 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b66fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b422f0 0x556772b67520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f5e444740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 0x556772b60360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f5e444740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556772b5ea20 con 0x556772b419d0 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f5e444740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556772b5e350 con 0x556772b4cbb0 2024-09-05T02:02:33.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f5e444740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556772b5f0f0 con 0x556772b422f0 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f5e444740 10 client.0 _ll_register_callbacks cb 0x7f0f50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59606640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b66fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59606640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b66fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58768/0 (socket says 172.21.15.103:58768) 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59606640 1 -- 172.21.15.103:0/2454762835 learned_addr learned my addr 172.21.15.103:0/2454762835 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59e07640 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b422f0 0x556772b67520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59606640 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 0x556772b60360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59606640 1 -- 172.21.15.103:0/2454762835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b422f0 msgr2=0x556772b67520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59606640 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b422f0 0x556772b67520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59606640 1 -- 172.21.15.103:0/2454762835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 msgr2=0x556772b60360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59606640 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 0x556772b60360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.564+0000 7f0f59606640 1 -- 172.21.15.103:0/2454762835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556772b60b70 con 0x556772b419d0 2024-09-05T02:02:33.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f59606640 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b66fe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800bb80 tx=0x7f0f4800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a682773b64bc600a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/2454762835 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48022040 con 0x556772b419d0 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/2454762835 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0f48011670 con 0x556772b419d0 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f5e444740 1 -- 172.21.15.103:0/2454762835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f50048040 con 0x556772b419d0 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/2454762835 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0f48011870 con 0x556772b419d0 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f5e444740 10 client.15273 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f5e444740 1 -- 172.21.15.103:0/2454762835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556772b65510 con 0x556772b419d0 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/2454762835 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f50048040 con 0x556772b419d0 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/2454762835 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f0f4802bc30 con 0x556772b419d0 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f5e444740 10 client.15273 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f5e444740 10 client.15273 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T02:02:33.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f5e444740 20 client.15273 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:33.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x556772bcc0a0 0x556772bcadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:33.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f5e444740 4 client.15273 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:33.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.565+0000 7f0f5e444740 1 -- 172.21.15.103:0/2454762835 --> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] -- command(tid 0: {"prefix": "status"}) -- 0x556772aafb10 con 0x556772bcc0a0 2024-09-05T02:02:33.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.566+0000 7f0f59e07640 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x556772bcc0a0 0x556772bcadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:33.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.566+0000 7f0f59e07640 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x556772bcc0a0 0x556772bcadb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd95baae8bd2a54f server_cookie=54d8879d53fe29a8 in_seq=0 out_seq=0 2024-09-05T02:02:33.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.566+0000 7f0f3d7fa640 10 client.15273 ms_handle_connect on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:33.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.566+0000 7f0f3d7fa640 1 -- 172.21.15.103:0/2454762835 <== mds.2 v2:172.21.15.139:6836/2336570561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556772aafb10 con 0x556772bcc0a0 2024-09-05T02:02:33.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.566+0000 7f0f3d7fa640 10 client.15273 handle_command_reply: tid=0 2024-09-05T02:02:33.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.566+0000 7f0f5e444740 1 client.15273 shutdown 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 -- 172.21.15.103:0/2454762835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 msgr2=0x556772b66fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b66fe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800bb80 tx=0x7f0f4800f730 comp rx=0 tx=0).stop 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 -- 172.21.15.103:0/2454762835 shutdown_connections 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556772b4cbb0 0x556772b60360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x556772bcc0a0 0x556772bcadb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556772b422f0 0x556772b67520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 --2- 172.21.15.103:0/2454762835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556772b419d0 0x556772b66fe0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f3d7fa640 0 client.15273 ms_handle_reset on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 -- 172.21.15.103:0/2454762835 >> 172.21.15.103:0/2454762835 conn(0x556772b3df60 msgr2=0x556772b3c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:33.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 -- 172.21.15.103:0/2454762835 shutdown_connections 2024-09-05T02:02:33.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 1 -- 172.21.15.103:0/2454762835 wait complete. 2024-09-05T02:02:33.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5e444740 20 client.15273 trim_cache size 0 max 16384 2024-09-05T02:02:33.579 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-05T02:02:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24464, 2024-09-05T02:02:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 97.370207527999995, 2024-09-05T02:02:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:02:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 162.613323087 2024-09-05T02:02:33.580 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:33.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0f400a2b80 msgr2=0x7f0f400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:33.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0f400a2b80 0x7f0f400a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0a44d0 tx=0x7f0f4c020000 comp rx=0 tx=0).stop 2024-09-05T02:02:33.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54102ef0 msgr2=0x7f0f54197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:33.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.567+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54102ef0 0x7f0f54197a70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00cce0 tx=0x7f0f4c00a620 comp rx=0 tx=0).stop 2024-09-05T02:02:33.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.568+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4204613566 shutdown_connections 2024-09-05T02:02:33.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.568+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f54102ef0 0x7f0f54197a70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.568+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f54102570 0x7f0f54197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.568+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0f400a2b80 0x7f0f400a5040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.568+0000 7f0f5c06b640 1 --2- 172.21.15.103:0/4204613566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f54100e20 0x7f0f54196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:33.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.568+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4204613566 >> 172.21.15.103:0/4204613566 conn(0x7f0f540fc7d0 msgr2=0x7f0f540fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:33.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.568+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4204613566 shutdown_connections 2024-09-05T02:02:33.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:33.568+0000 7f0f5c06b640 1 -- 172.21.15.103:0/4204613566 wait complete. 2024-09-05T02:02:34.269 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T02:02:34.526 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:34.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:34 smithi103 ceph-mon[30902]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 895 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:34 smithi167 ceph-mon[35122]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 895 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:34.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:34 smithi139 ceph-mon[34262]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 895 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:35.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.034+0000 7f013cb12640 1 -- 172.21.15.103:0/4173988695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 msgr2=0x7f013810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:35.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.034+0000 7f013cb12640 1 --2- 172.21.15.103:0/4173988695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 0x7f013810bb50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0120009900 tx=0x7f012002a480 comp rx=0 tx=0).stop 2024-09-05T02:02:35.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.034+0000 7f013cb12640 1 -- 172.21.15.103:0/4173988695 shutdown_connections 2024-09-05T02:02:35.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.034+0000 7f013cb12640 1 --2- 172.21.15.103:0/4173988695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f013810c170 0x7f013810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.034+0000 7f013cb12640 1 --2- 172.21.15.103:0/4173988695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 0x7f013810bb50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.034+0000 7f013cb12640 1 --2- 172.21.15.103:0/4173988695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0138102d40 0x7f0138103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.034+0000 7f013cb12640 1 -- 172.21.15.103:0/4173988695 >> 172.21.15.103:0/4173988695 conn(0x7f01380fc7a0 msgr2=0x7f01380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.034+0000 7f013cb12640 1 -- 172.21.15.103:0/4173988695 shutdown_connections 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.034+0000 7f013cb12640 1 -- 172.21.15.103:0/4173988695 wait complete. 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.035+0000 7f013cb12640 1 Processor -- start 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.035+0000 7f013cb12640 1 -- start start 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.035+0000 7f013cb12640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0138102d40 0x7f0138198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.035+0000 7f013cb12640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 0x7f01381988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.035+0000 7f013cb12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f013810c170 0x7f013819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.035+0000 7f013cb12640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0138111e20 con 0x7f0138103660 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.035+0000 7f013cb12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0138111ca0 con 0x7f013810c170 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.035+0000 7f013cb12640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0138111fa0 con 0x7f0138102d40 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0135d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 0x7f01381988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f013659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0138102d40 0x7f0138198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0135d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 0x7f01381988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58788/0 (socket says 172.21.15.103:58788) 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0135d9b640 1 -- 172.21.15.103:0/933433888 learned_addr learned my addr 172.21.15.103:0/933433888 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0136d9d640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f013810c170 0x7f013819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0135d9b640 1 -- 172.21.15.103:0/933433888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0138102d40 msgr2=0x7f0138198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0135d9b640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0138102d40 0x7f0138198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0135d9b640 1 -- 172.21.15.103:0/933433888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f013810c170 msgr2=0x7f013819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0135d9b640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f013810c170 0x7f013819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0135d9b640 1 -- 172.21.15.103:0/933433888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013819d330 con 0x7f0138103660 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f0135d9b640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 0x7f01381988c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0120005940 tx=0x7f01200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f012f7fe640 1 -- 172.21.15.103:0/933433888 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012003f030 con 0x7f0138103660 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f012f7fe640 1 -- 172.21.15.103:0/933433888 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0120003cb0 con 0x7f0138103660 2024-09-05T02:02:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f013cb12640 1 -- 172.21.15.103:0/933433888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013819d5c0 con 0x7f0138103660 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f012f7fe640 1 -- 172.21.15.103:0/933433888 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120035070 con 0x7f0138103660 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.036+0000 7f013cb12640 1 -- 172.21.15.103:0/933433888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01381a4f00 con 0x7f0138103660 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.039+0000 7f012f7fe640 1 -- 172.21.15.103:0/933433888 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0120042030 con 0x7f0138103660 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.039+0000 7f013eeeb740 1 Processor -- start 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f013eeeb740 1 -- start start 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5c9d0 0x563e10c5cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5d2f0 0x563e10c67670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 0x563e10c69fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f013eeeb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e10bc3090 con 0x563e10c5c9d0 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f013eeeb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563e10b6aa70 con 0x563e10c5d2f0 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f013eeeb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563e10bbf050 con 0x563e10c67bb0 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0136d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5c9d0 0x563e10c5cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0136d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5c9d0 0x563e10c5cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58798/0 (socket says 172.21.15.103:58798) 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0136d9d640 1 -- 172.21.15.103:0/1821793342 learned_addr learned my addr 172.21.15.103:0/1821793342 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0135d9b640 1 --2- 172.21.15.103:0/1821793342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 0x563e10c69fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f013659c640 1 --2- 172.21.15.103:0/1821793342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5d2f0 0x563e10c67670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f012f7fe640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f01240a2b80 0x7f01240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0136d9d640 1 -- 172.21.15.103:0/1821793342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 msgr2=0x563e10c69fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0136d9d640 1 --2- 172.21.15.103:0/1821793342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 0x563e10c69fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0136d9d640 1 -- 172.21.15.103:0/1821793342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5d2f0 msgr2=0x563e10c67670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0136d9d640 1 --2- 172.21.15.103:0/1821793342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5d2f0 0x563e10c67670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0136d9d640 1 -- 172.21.15.103:0/1821793342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e10b77130 con 0x563e10c5c9d0 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.040+0000 7f0136d9d640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f01240a2b80 0x7f01240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.041+0000 7f012f7fe640 1 -- 172.21.15.103:0/933433888 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f01200efb60 con 0x7f0138103660 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.041+0000 7f0136d9d640 1 --2- 172.21.15.103:0/1821793342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5c9d0 0x563e10c5cdb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f012800b7a0 tx=0x7f012800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c483eddc50efa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.041+0000 7f012cff9640 1 -- 172.21.15.103:0/1821793342 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128024070 con 0x563e10c5c9d0 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.041+0000 7f012cff9640 1 -- 172.21.15.103:0/1821793342 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f012801e930 con 0x563e10c5c9d0 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.041+0000 7f012cff9640 1 -- 172.21.15.103:0/1821793342 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128011860 con 0x563e10c5c9d0 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.041+0000 7f0136d9d640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f01240a2b80 0x7f01240a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0128021000 tx=0x7f012801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 -- 172.21.15.103:0/1821793342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5c9d0 msgr2=0x563e10c5cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1821793342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5c9d0 0x563e10c5cdb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f012800b7a0 tx=0x7f012800bc60 comp rx=0 tx=0).stop 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 -- 172.21.15.103:0/1821793342 shutdown_connections 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1821793342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 0x563e10c69fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1821793342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5d2f0 0x563e10c67670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1821793342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5c9d0 0x563e10c5cdb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 -- 172.21.15.103:0/1821793342 >> 172.21.15.103:0/1821793342 conn(0x563e10c58f60 msgr2=0x563e10c59340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 -- 172.21.15.103:0/1821793342 shutdown_connections 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 -- 172.21.15.103:0/1821793342 wait complete. 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 -- start start 2024-09-05T02:02:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 Processor -- start 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 0x563e10c67470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 0x563e10c64d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e10c6a780 con 0x563e10c5d2f0 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563e10b7df20 con 0x563e10c67bb0 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563e10c6a900 con 0x563e10c5c9d0 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f013eeeb740 10 client.0 _ll_register_callbacks cb 0x563e10b47030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f0135d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f0135d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58802/0 (socket says 172.21.15.103:58802) 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.042+0000 7f0135d9b640 1 -- 172.21.15.103:0/3490843818 learned_addr learned my addr 172.21.15.103:0/3490843818 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f0136d9d640 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 0x563e10c64d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f013659c640 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 0x563e10c67470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f0135d9b640 1 -- 172.21.15.103:0/3490843818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 msgr2=0x563e10c67470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f0135d9b640 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 0x563e10c67470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f0135d9b640 1 -- 172.21.15.103:0/3490843818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 msgr2=0x563e10c64d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f0135d9b640 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 0x563e10c64d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f0135d9b640 1 -- 172.21.15.103:0/3490843818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e109342b0 con 0x563e10c5d2f0 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f0136d9d640 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 0x563e10c64d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f0135d9b640 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c647c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0120006010 tx=0x7f0120009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bbcf55ad665d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f011dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f011dffb640 1 -- 172.21.15.103:0/3490843818 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120049070 con 0x563e10c5d2f0 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f011dffb640 1 -- 172.21.15.103:0/3490843818 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f01200b8730 con 0x563e10c5d2f0 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f013eeeb740 1 -- 172.21.15.103:0/3490843818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012801e930 con 0x563e10c5d2f0 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f011dffb640 1 -- 172.21.15.103:0/3490843818 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f01200bba20 con 0x563e10c5d2f0 2024-09-05T02:02:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f013eeeb740 10 client.15285 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f013eeeb740 1 -- 172.21.15.103:0/3490843818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e10c6a780 con 0x563e10c5d2f0 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.043+0000 7f011dffb640 1 -- 172.21.15.103:0/3490843818 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f012801e930 con 0x563e10c5d2f0 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f011dffb640 1 -- 172.21.15.103:0/3490843818 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f0120003800 con 0x563e10c5d2f0 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f013eeeb740 10 client.15285 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f013eeeb740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f013eeeb740 20 client.15285 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x563e10c78710 0x563e10c79b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f013eeeb740 4 client.15285 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f013eeeb740 1 -- 172.21.15.103:0/3490843818 --> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e10bd0600 con 0x563e10c78710 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f013659c640 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x563e10c78710 0x563e10c79b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f013659c640 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x563e10c78710 0x563e10c79b80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91856dffd7de2cd0 server_cookie=9309d12406de42c6 in_seq=0 out_seq=0 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.044+0000 7f011dffb640 10 client.15285 ms_handle_connect on v2:172.21.15.167:6838/1858062899 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f011dffb640 1 -- 172.21.15.103:0/3490843818 <== mds.1 v2:172.21.15.167:6838/1858062899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e10bd0600 con 0x563e10c78710 2024-09-05T02:02:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f011dffb640 10 client.15285 handle_command_reply: tid=0 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f013eeeb740 1 client.15285 shutdown 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f013eeeb740 1 -- 172.21.15.103:0/3490843818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 msgr2=0x563e10c647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c647c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0120006010 tx=0x7f0120009620 comp rx=0 tx=0).stop 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f013eeeb740 1 -- 172.21.15.103:0/3490843818 shutdown_connections 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x563e10c78710 0x563e10c79b80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 0x563e10c64d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f011dffb640 0 client.15285 ms_handle_reset on v2:172.21.15.167:6838/1858062899 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c647c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3490843818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 0x563e10c67470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.045+0000 7f013eeeb740 1 -- 172.21.15.103:0/3490843818 >> 172.21.15.103:0/3490843818 conn(0x563e10c58f60 msgr2=0x563e10c5ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.046+0000 7f013eeeb740 1 -- 172.21.15.103:0/3490843818 shutdown_connections 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.046+0000 7f013eeeb740 1 -- 172.21.15.103:0/3490843818 wait complete. 2024-09-05T02:02:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.046+0000 7f013eeeb740 20 client.15285 trim_cache size 0 max 16384 2024-09-05T02:02:35.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.063+0000 7f013eeeb740 1 Processor -- start 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.063+0000 7f013eeeb740 1 -- start start 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.063+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 0x563e10c64080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.063+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c77250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.063+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 0x563e10c77790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.063+0000 7f013eeeb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e10c77ec0 con 0x563e10c5d2f0 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.063+0000 7f013eeeb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563e10c78590 con 0x563e10c67bb0 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.063+0000 7f013eeeb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563e10c78c60 con 0x563e10c5c9d0 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0135d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c77250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0135d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c77250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58810/0 (socket says 172.21.15.103:58810) 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0135d9b640 1 -- 172.21.15.103:0/3265831700 learned_addr learned my addr 172.21.15.103:0/3265831700 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f013659c640 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 0x563e10c64080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0136d9d640 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 0x563e10c77790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0135d9b640 1 -- 172.21.15.103:0/3265831700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 msgr2=0x563e10c64080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0135d9b640 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 0x563e10c64080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0135d9b640 1 -- 172.21.15.103:0/3265831700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 msgr2=0x563e10c77790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0135d9b640 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 0x563e10c77790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0135d9b640 1 -- 172.21.15.103:0/3265831700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e10c5d120 con 0x563e10c5d2f0 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f013659c640 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 0x563e10c64080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f0135d9b640 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c77250 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f01200b8520 tx=0x7f0120005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0741243e6e8d5d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f011ffff640 1 -- 172.21.15.103:0/3265831700 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120049070 con 0x563e10c5d2f0 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f011ffff640 1 -- 172.21.15.103:0/3265831700 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01200bc0c0 con 0x563e10c5d2f0 2024-09-05T02:02:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.064+0000 7f011ffff640 1 -- 172.21.15.103:0/3265831700 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120005d50 con 0x563e10c5d2f0 2024-09-05T02:02:35.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.065+0000 7f013eeeb740 1 -- 172.21.15.103:0/3265831700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 msgr2=0x563e10c77250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:35.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.065+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c77250 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f01200b8520 tx=0x7f0120005c90 comp rx=0 tx=0).stop 2024-09-05T02:02:35.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.065+0000 7f013eeeb740 1 -- 172.21.15.103:0/3265831700 shutdown_connections 2024-09-05T02:02:35.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.065+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c67bb0 0x563e10c77790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.065+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c77250 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.065+0000 7f013eeeb740 1 --2- 172.21.15.103:0/3265831700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c5c9d0 0x563e10c64080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.065+0000 7f013eeeb740 1 -- 172.21.15.103:0/3265831700 >> 172.21.15.103:0/3265831700 conn(0x563e10c58f60 msgr2=0x563e10c63090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.065+0000 7f013eeeb740 1 -- 172.21.15.103:0/3265831700 shutdown_connections 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.065+0000 7f013eeeb740 1 -- 172.21.15.103:0/3265831700 wait complete. 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013eeeb740 1 -- start start 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013eeeb740 1 Processor -- start 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5c9d0 0x563e10c822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c7d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013eeeb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 0x563e10c7db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013eeeb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e10c79a00 con 0x563e10c5d2f0 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013eeeb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563e10c79330 con 0x563e10c5c9d0 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013eeeb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563e10c7a0d0 con 0x563e10c67bb0 2024-09-05T02:02:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013eeeb740 10 client.0 _ll_register_callbacks cb 0x7f01200f3330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f0135d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c7d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f0135d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c7d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58822/0 (socket says 172.21.15.103:58822) 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f0135d9b640 1 -- 172.21.15.103:0/1243765121 learned_addr learned my addr 172.21.15.103:0/1243765121 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013659c640 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5c9d0 0x563e10c822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f0136d9d640 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 0x563e10c7db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f0135d9b640 1 -- 172.21.15.103:0/1243765121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 msgr2=0x563e10c7db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f0135d9b640 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 0x563e10c7db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f0135d9b640 1 -- 172.21.15.103:0/1243765121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5c9d0 msgr2=0x563e10c822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f0135d9b640 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5c9d0 0x563e10c822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f0135d9b640 1 -- 172.21.15.103:0/1243765121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e10c7e070 con 0x563e10c5d2f0 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.066+0000 7f013659c640 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5c9d0 0x563e10c822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:02:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f0135d9b640 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c7d5f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f01200c0480 tx=0x7f0120003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66926f47154730fb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f011dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f011dffb640 1 -- 172.21.15.103:0/1243765121 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01200d4020 con 0x563e10c5d2f0 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f013eeeb740 1 -- 172.21.15.103:0/1243765121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01200bc0c0 con 0x563e10c5d2f0 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f011dffb640 1 -- 172.21.15.103:0/1243765121 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f01200c47f0 con 0x563e10c5d2f0 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f011dffb640 1 -- 172.21.15.103:0/1243765121 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0120049070 con 0x563e10c5d2f0 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f013eeeb740 10 client.15291 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f013eeeb740 1 -- 172.21.15.103:0/1243765121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e10c7e070 con 0x563e10c5d2f0 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f011dffb640 1 -- 172.21.15.103:0/1243765121 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01200bc0c0 con 0x563e10c5d2f0 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f011dffb640 1 -- 172.21.15.103:0/1243765121 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f01200bb5a0 con 0x563e10c5d2f0 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f013eeeb740 10 client.15291 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.067+0000 7f013eeeb740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.068+0000 7f013eeeb740 20 client.15291 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.068+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x563e10ce6fc0 0x563e10ce5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.068+0000 7f013eeeb740 4 client.15291 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.068+0000 7f013eeeb740 1 -- 172.21.15.103:0/1243765121 --> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] -- command(tid 0: {"prefix": "status"}) -- 0x563e10bcab10 con 0x563e10ce6fc0 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.068+0000 7f013659c640 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x563e10ce6fc0 0x563e10ce5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.068+0000 7f013659c640 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x563e10ce6fc0 0x563e10ce5cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d9ca92af9dbb085 server_cookie=3e2fafefe0d16e76 in_seq=0 out_seq=0 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.068+0000 7f011dffb640 10 client.15291 ms_handle_connect on v2:172.21.15.167:6838/1858062899 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f011dffb640 1 -- 172.21.15.103:0/1243765121 <== mds.1 v2:172.21.15.167:6838/1858062899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563e10bcab10 con 0x563e10ce6fc0 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f011dffb640 10 client.15291 handle_command_reply: tid=0 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 client.15291 shutdown 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 -- 172.21.15.103:0/1243765121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 msgr2=0x563e10c7d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c7d5f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f01200c0480 tx=0x7f0120003ed0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 -- 172.21.15.103:0/1243765121 shutdown_connections 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x563e10ce6fc0 0x563e10ce5cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e10c67bb0 0x563e10c7db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e10c5d2f0 0x563e10c7d5f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f011dffb640 0 client.15291 ms_handle_reset on v2:172.21.15.167:6838/1858062899 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 --2- 172.21.15.103:0/1243765121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563e10c5c9d0 0x563e10c822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 -- 172.21.15.103:0/1243765121 >> 172.21.15.103:0/1243765121 conn(0x563e10c58f60 msgr2=0x563e10c77250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 -- 172.21.15.103:0/1243765121 shutdown_connections 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 1 -- 172.21.15.103:0/1243765121 wait complete. 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.069+0000 7f013eeeb740 20 client.15291 trim_cache size 0 max 16384 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-05T02:02:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24466, 2024-09-05T02:02:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 99.876930762000001, 2024-09-05T02:02:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:02:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 161.511588674 2024-09-05T02:02:35.077 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:35.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 -- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f01240a2b80 msgr2=0x7f01240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:35.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f01240a2b80 0x7f01240a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0128021000 tx=0x7f012801e000 comp rx=0 tx=0).stop 2024-09-05T02:02:35.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 -- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 msgr2=0x7f01381988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:35.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 0x7f01381988c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0120005940 tx=0x7f01200042f0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 -- 172.21.15.103:0/933433888 shutdown_connections 2024-09-05T02:02:35.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f013810c170 0x7f013819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138103660 0x7f01381988c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f01240a2b80 0x7f01240a5040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 --2- 172.21.15.103:0/933433888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0138102d40 0x7f0138198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:35.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 -- 172.21.15.103:0/933433888 >> 172.21.15.103:0/933433888 conn(0x7f01380fc7a0 msgr2=0x7f01380fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:35.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 -- 172.21.15.103:0/933433888 shutdown_connections 2024-09-05T02:02:35.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:35.070+0000 7f013cb12640 1 -- 172.21.15.103:0/933433888 wait complete. 2024-09-05T02:02:35.772 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T02:02:36.020 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:36.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:36 smithi103 ceph-mon[30902]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 688 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:36.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.518+0000 7f6b23fc3640 1 -- 172.21.15.103:0/3704437957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c1027a0 msgr2=0x7f6b1c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:36.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.518+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/3704437957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c1027a0 0x7f6b1c1067f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009900 tx=0x7f6b0c02a410 comp rx=0 tx=0).stop 2024-09-05T02:02:36.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.518+0000 7f6b23fc3640 1 -- 172.21.15.103:0/3704437957 shutdown_connections 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.518+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/3704437957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b1c106f20 0x7f6b1c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.518+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/3704437957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c1027a0 0x7f6b1c1067f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.518+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/3704437957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b1c101df0 0x7f6b1c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.518+0000 7f6b23fc3640 1 -- 172.21.15.103:0/3704437957 >> 172.21.15.103:0/3704437957 conn(0x7f6b1c0fd8b0 msgr2=0x7f6b1c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.518+0000 7f6b23fc3640 1 -- 172.21.15.103:0/3704437957 shutdown_connections 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.518+0000 7f6b23fc3640 1 -- 172.21.15.103:0/3704437957 wait complete. 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b23fc3640 1 Processor -- start 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b23fc3640 1 -- start start 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b23fc3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b1c101df0 0x7f6b1c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b23fc3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b1c1027a0 0x7f6b1c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b23fc3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c106f20 0x7f6b1c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b23fc3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c111e30 con 0x7f6b1c106f20 2024-09-05T02:02:36.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b23fc3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c111cb0 con 0x7f6b1c1027a0 2024-09-05T02:02:36.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b23fc3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c111fb0 con 0x7f6b1c101df0 2024-09-05T02:02:36.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b22560640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c106f20 0x7f6b1c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b22560640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c106f20 0x7f6b1c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58848/0 (socket says 172.21.15.103:58848) 2024-09-05T02:02:36.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b22560640 1 -- 172.21.15.103:0/1061807016 learned_addr learned my addr 172.21.15.103:0/1061807016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:36.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b1c101df0 0x7f6b1c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.519+0000 7f6b2155e640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b1c1027a0 0x7f6b1c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b22560640 1 -- 172.21.15.103:0/1061807016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b1c101df0 msgr2=0x7f6b1c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b22560640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b1c101df0 0x7f6b1c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b22560640 1 -- 172.21.15.103:0/1061807016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b1c1027a0 msgr2=0x7f6b1c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b22560640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b1c1027a0 0x7f6b1c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b22560640 1 -- 172.21.15.103:0/1061807016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c19d360 con 0x7f6b1c106f20 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b22560640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c106f20 0x7f6b1c19cc80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400eb10 tx=0x7f6b1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b12ffd640 1 -- 172.21.15.103:0/1061807016 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14004b20 con 0x7f6b1c106f20 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b23fc3640 1 -- 172.21.15.103:0/1061807016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c19d650 con 0x7f6b1c106f20 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b12ffd640 1 -- 172.21.15.103:0/1061807016 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b14004cc0 con 0x7f6b1c106f20 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b12ffd640 1 -- 172.21.15.103:0/1061807016 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14010690 con 0x7f6b1c106f20 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.520+0000 7f6b23fc3640 1 -- 172.21.15.103:0/1061807016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c1a4f30 con 0x7f6b1c106f20 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.522+0000 7f6b12ffd640 1 -- 172.21.15.103:0/1061807016 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6b14005020 con 0x7f6b1c106f20 2024-09-05T02:02:36.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2639c740 1 Processor -- start 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2639c740 1 -- start start 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b12ffd640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b080a2b80 0x7f6b080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed28d0 0x562f46ed2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed31f0 0x562f46ee1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46ee43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2639c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f46e39090 con 0x562f46ed28d0 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2639c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562f46de0a70 con 0x562f46ed31f0 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2639c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f46e35050 con 0x562f46ee1fc0 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2155e640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b080a2b80 0x7f6b080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b12ffd640 1 -- 172.21.15.103:0/1061807016 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6b140cfb00 con 0x7f6b1c106f20 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b21d5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed28d0 0x562f46ed2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b21d5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed28d0 0x562f46ed2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58850/0 (socket says 172.21.15.103:58850) 2024-09-05T02:02:36.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b21d5f640 1 -- 172.21.15.103:0/3152427745 learned_addr learned my addr 172.21.15.103:0/3152427745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b2155e640 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed31f0 0x562f46ee1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.523+0000 7f6b22560640 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46ee43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b21d5f640 1 -- 172.21.15.103:0/3152427745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 msgr2=0x562f46ee43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46ee43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b21d5f640 1 -- 172.21.15.103:0/3152427745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed31f0 msgr2=0x562f46ee1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed31f0 0x562f46ee1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b21d5f640 1 -- 172.21.15.103:0/3152427745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f46ded130 con 0x562f46ed28d0 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b2155e640 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed31f0 0x562f46ee1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b2155e640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b080a2b80 0x7f6b080a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c031440 tx=0x7f6b0c005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed28d0 0x562f46ed2cb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009cf0 tx=0x7f6b18011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c674ee8134034af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:36.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b07fff640 1 -- 172.21.15.103:0/3152427745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18009d20 con 0x562f46ed28d0 2024-09-05T02:02:36.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b07fff640 1 -- 172.21.15.103:0/3152427745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b18005020 con 0x562f46ed28d0 2024-09-05T02:02:36.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.524+0000 7f6b07fff640 1 -- 172.21.15.103:0/3152427745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18004b20 con 0x562f46ed28d0 2024-09-05T02:02:36.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 -- 172.21.15.103:0/3152427745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed28d0 msgr2=0x562f46ed2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:36.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed28d0 0x562f46ed2cb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009cf0 tx=0x7f6b18011810 comp rx=0 tx=0).stop 2024-09-05T02:02:36.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 -- 172.21.15.103:0/3152427745 shutdown_connections 2024-09-05T02:02:36.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46ee43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed31f0 0x562f46ee1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 --2- 172.21.15.103:0/3152427745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed28d0 0x562f46ed2cb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 -- 172.21.15.103:0/3152427745 >> 172.21.15.103:0/3152427745 conn(0x562f46ecef60 msgr2=0x562f46ecf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 -- 172.21.15.103:0/3152427745 shutdown_connections 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 -- 172.21.15.103:0/3152427745 wait complete. 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 -- start start 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.525+0000 7f6b2639c740 1 Processor -- start 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ed28d0 0x562f46ee1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ee1fc0 0x562f46edf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2639c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f46ee4bf0 con 0x562f46ed31f0 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2639c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562f46df3f20 con 0x562f46ee1fc0 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2639c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f46ee4d70 con 0x562f46ed28d0 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2639c740 10 client.0 _ll_register_callbacks cb 0x562f46dbd030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:36.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2155e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2155e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58854/0 (socket says 172.21.15.103:58854) 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2155e640 1 -- 172.21.15.103:0/1386476661 learned_addr learned my addr 172.21.15.103:0/1386476661 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b22560640 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ee1fc0 0x562f46edf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ed28d0 0x562f46ee1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2155e640 1 -- 172.21.15.103:0/1386476661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ed28d0 msgr2=0x562f46ee1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2155e640 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ed28d0 0x562f46ee1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2155e640 1 -- 172.21.15.103:0/1386476661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ee1fc0 msgr2=0x562f46edf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2155e640 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ee1fc0 0x562f46edf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2155e640 1 -- 172.21.15.103:0/1386476661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f46baa2b0 con 0x562f46ed31f0 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b2155e640 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edeba0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009900 tx=0x7f6b0c045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9b8baec772649d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.526+0000 7f6b057fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:36.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b057fa640 1 -- 172.21.15.103:0/1386476661 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c04e070 con 0x562f46ed31f0 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b057fa640 1 -- 172.21.15.103:0/1386476661 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6b0c005270 con 0x562f46ed31f0 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b2639c740 1 -- 172.21.15.103:0/1386476661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b18005020 con 0x562f46ed31f0 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b057fa640 1 -- 172.21.15.103:0/1386476661 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b0c05a050 con 0x562f46ed31f0 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b2639c740 10 client.15303 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b2639c740 1 -- 172.21.15.103:0/1386476661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f46ef0540 con 0x562f46ed31f0 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b057fa640 1 -- 172.21.15.103:0/1386476661 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b18005020 con 0x562f46ed31f0 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b057fa640 1 -- 172.21.15.103:0/1386476661 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6b0c03a040 con 0x562f46ed31f0 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b2639c740 10 client.15303 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b2639c740 10 client.15303 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b2639c740 20 client.15303 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:36.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b2639c740 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562f46ef2bf0 0x562f46ef4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b2639c740 4 client.15303 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b2639c740 1 -- 172.21.15.103:0/1386476661 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f46e46600 con 0x562f46ef2bf0 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.527+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562f46ef2bf0 0x562f46ef4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.528+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562f46ef2bf0 0x562f46ef4060 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7a601da4929658 server_cookie=d4b9d4ac158ee869 in_seq=0 out_seq=0 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.528+0000 7f6b057fa640 10 client.15303 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.528+0000 7f6b057fa640 1 -- 172.21.15.103:0/1386476661 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f46e46600 con 0x562f46ef2bf0 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.528+0000 7f6b057fa640 10 client.15303 handle_command_reply: tid=0 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 client.15303 shutdown 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 -- 172.21.15.103:0/1386476661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 msgr2=0x562f46edeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edeba0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009900 tx=0x7f6b0c045000 comp rx=0 tx=0).stop 2024-09-05T02:02:36.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 -- 172.21.15.103:0/1386476661 shutdown_connections 2024-09-05T02:02:36.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562f46ef2bf0 0x562f46ef4060 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ee1fc0 0x562f46edf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edeba0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b057fa640 0 client.15303 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:36.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 --2- 172.21.15.103:0/1386476661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ed28d0 0x562f46ee1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 -- 172.21.15.103:0/1386476661 >> 172.21.15.103:0/1386476661 conn(0x562f46ecef60 msgr2=0x562f46ed4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:36.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 -- 172.21.15.103:0/1386476661 shutdown_connections 2024-09-05T02:02:36.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 1 -- 172.21.15.103:0/1386476661 wait complete. 2024-09-05T02:02:36.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.529+0000 7f6b2639c740 20 client.15303 trim_cache size 0 max 16384 2024-09-05T02:02:36.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2639c740 1 Processor -- start 2024-09-05T02:02:36.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2639c740 1 -- start start 2024-09-05T02:02:36.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 0x562f46ef4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46ede460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46edd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2639c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f46ef23a0 con 0x562f46ed31f0 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2639c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562f46ef2a70 con 0x562f46ed28d0 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2639c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f46ef3140 con 0x562f46ee1fc0 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2155e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46ede460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2155e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46ede460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58866/0 (socket says 172.21.15.103:58866) 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2155e640 1 -- 172.21.15.103:0/2340212343 learned_addr learned my addr 172.21.15.103:0/2340212343 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/2340212343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 0x562f46ef4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2155e640 1 -- 172.21.15.103:0/2340212343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 msgr2=0x562f46edd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2155e640 1 --2- 172.21.15.103:0/2340212343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46edd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2155e640 1 -- 172.21.15.103:0/2340212343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 msgr2=0x562f46ef4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2155e640 1 --2- 172.21.15.103:0/2340212343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 0x562f46ef4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.547+0000 7f6b2155e640 1 -- 172.21.15.103:0/2340212343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f46edf960 con 0x562f46ed31f0 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b2155e640 1 --2- 172.21.15.103:0/2340212343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46ede460 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c032f10 tx=0x7f6b0c032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509fc1770e2f2ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b077fe640 1 -- 172.21.15.103:0/2340212343 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c04e040 con 0x562f46ed31f0 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b077fe640 1 -- 172.21.15.103:0/2340212343 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b0c0035f0 con 0x562f46ed31f0 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b077fe640 1 -- 172.21.15.103:0/2340212343 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c035040 con 0x562f46ed31f0 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b2639c740 1 -- 172.21.15.103:0/2340212343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 msgr2=0x562f46ede460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2340212343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46ede460 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c032f10 tx=0x7f6b0c032fc0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b2639c740 1 -- 172.21.15.103:0/2340212343 shutdown_connections 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2340212343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46edd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2340212343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46ede460 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.548+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2340212343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 0x562f46ef4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.549+0000 7f6b2639c740 1 -- 172.21.15.103:0/2340212343 >> 172.21.15.103:0/2340212343 conn(0x562f46ecef60 msgr2=0x562f46ecf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.549+0000 7f6b2639c740 1 -- 172.21.15.103:0/2340212343 shutdown_connections 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.549+0000 7f6b2639c740 1 -- 172.21.15.103:0/2340212343 wait complete. 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.549+0000 7f6b2639c740 1 -- start start 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.549+0000 7f6b2639c740 1 Processor -- start 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.549+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 0x562f46edb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.549+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.549+0000 7f6b2639c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46efb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.549+0000 7f6b2639c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f46ef3ee0 con 0x562f46ed31f0 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2639c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562f46ef3810 con 0x562f46ed28d0 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2639c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f46ef45b0 con 0x562f46ee1fc0 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2639c740 10 client.0 _ll_register_callbacks cb 0x7f6b140093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:36.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2155e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2155e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58876/0 (socket says 172.21.15.103:58876) 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2155e640 1 -- 172.21.15.103:0/2313978441 learned_addr learned my addr 172.21.15.103:0/2313978441 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b22560640 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46efb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 0x562f46edb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2155e640 1 -- 172.21.15.103:0/2313978441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 msgr2=0x562f46efb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2155e640 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46efb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2155e640 1 -- 172.21.15.103:0/2313978441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 msgr2=0x562f46edb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2155e640 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 0x562f46edb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2155e640 1 -- 172.21.15.103:0/2313978441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f46efb7f0 con 0x562f46ed31f0 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 0x562f46edb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:36.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2155e640 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edb710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c02a930 tx=0x7f6b0c031d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47d126b8c974d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b057fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b057fa640 1 -- 172.21.15.103:0/2313978441 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c035040 con 0x562f46ed31f0 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b2639c740 1 -- 172.21.15.103:0/2313978441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f46e40b10 con 0x562f46ed31f0 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b057fa640 1 -- 172.21.15.103:0/2313978441 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6b0c0046c0 con 0x562f46ed31f0 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.550+0000 7f6b057fa640 1 -- 172.21.15.103:0/2313978441 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b0c071050 con 0x562f46ed31f0 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b2639c740 10 client.15309 fetch_fsmap learned FSMap version 28 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b2639c740 1 -- 172.21.15.103:0/2313978441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f46efcaf0 con 0x562f46ed31f0 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b057fa640 1 -- 172.21.15.103:0/2313978441 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562f46e40b10 con 0x562f46ed31f0 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b057fa640 1 -- 172.21.15.103:0/2313978441 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6b0c0052d0 con 0x562f46ed31f0 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b2639c740 10 client.15309 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T02:02:36.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b2639c740 10 client.15309 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b2639c740 20 client.15309 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562f46f61270 0x562f46f5ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b2639c740 4 client.15309 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b2639c740 1 -- 172.21.15.103:0/2313978441 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "status"}) -- 0x562f46dbd0e0 con 0x562f46f61270 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.551+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562f46f61270 0x562f46f5ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b21d5f640 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562f46f61270 0x562f46f5ff80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b199ca670bdb99 server_cookie=77afd67c139736db in_seq=0 out_seq=0 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b057fa640 10 client.15309 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b057fa640 1 -- 172.21.15.103:0/2313978441 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562f46dbd0e0 con 0x562f46f61270 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b057fa640 10 client.15309 handle_command_reply: tid=0 2024-09-05T02:02:36.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 client.15309 shutdown 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 -- 172.21.15.103:0/2313978441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 msgr2=0x562f46edb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edb710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c02a930 tx=0x7f6b0c031d80 comp rx=0 tx=0).stop 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 -- 172.21.15.103:0/2313978441 shutdown_connections 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562f46f61270 0x562f46f5ff80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f46ee1fc0 0x562f46efb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f46ed31f0 0x562f46edb710 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b057fa640 0 client.15309 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 --2- 172.21.15.103:0/2313978441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f46ed28d0 0x562f46edb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 -- 172.21.15.103:0/2313978441 >> 172.21.15.103:0/2313978441 conn(0x562f46ecef60 msgr2=0x562f46ecf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 -- 172.21.15.103:0/2313978441 shutdown_connections 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 1 -- 172.21.15.103:0/2313978441 wait complete. 2024-09-05T02:02:36.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.552+0000 7f6b2639c740 20 client.15309 trim_cache size 0 max 16384 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24497, 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 141.184612483, 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 28, 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:02:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T02:02:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 152.870220128 2024-09-05T02:02:36.565 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:36.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 -- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b080a2b80 msgr2=0x7f6b080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:36.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b080a2b80 0x7f6b080a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c031440 tx=0x7f6b0c005e80 comp rx=0 tx=0).stop 2024-09-05T02:02:36.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 -- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c106f20 msgr2=0x7f6b1c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:36.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c106f20 0x7f6b1c19cc80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400eb10 tx=0x7f6b1400c2d0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 -- 172.21.15.103:0/1061807016 shutdown_connections 2024-09-05T02:02:36.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b1c106f20 0x7f6b1c19cc80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b1c1027a0 0x7f6b1c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6b080a2b80 0x7f6b080a5040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 --2- 172.21.15.103:0/1061807016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b1c101df0 0x7f6b1c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:36.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 -- 172.21.15.103:0/1061807016 >> 172.21.15.103:0/1061807016 conn(0x7f6b1c0fd8b0 msgr2=0x7f6b1c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:36.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 -- 172.21.15.103:0/1061807016 shutdown_connections 2024-09-05T02:02:36.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:36.553+0000 7f6b23fc3640 1 -- 172.21.15.103:0/1061807016 wait complete. 2024-09-05T02:02:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:36 smithi167 ceph-mon[35122]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 688 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:36.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:36 smithi139 ceph-mon[34262]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 688 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:37.291 INFO:teuthology.run_tasks:Running task sequential... 2024-09-05T02:02:37.302 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-05T02:02:37.302 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-05T02:02:37.303 INFO:teuthology.task.print:Enabling mgr modules 2024-09-05T02:02:37.303 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-05T02:02:37.365 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-05T02:02:37.366 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-05T02:02:37.366 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-05T02:02:37.366 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T02:02:37.366 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:37.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:37.366 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-05T02:02:37.384 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-05T02:02:37.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:37.384 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T02:02:37.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:37.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-05T02:02:37.551 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-05T02:02:37.551 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-05T02:02:37.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T02:02:37.797 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:38.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.289+0000 7f4bd17c6640 1 -- 172.21.15.103:0/3317805491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc101df0 msgr2=0x7f4bcc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.289+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/3317805491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc101df0 0x7f4bcc1021d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009900 tx=0x7f4bbc02a410 comp rx=0 tx=0).stop 2024-09-05T02:02:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.289+0000 7f4bd17c6640 1 -- 172.21.15.103:0/3317805491 shutdown_connections 2024-09-05T02:02:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.289+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/3317805491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc106f20 0x7f4bcc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.289+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/3317805491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc1027a0 0x7f4bcc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.289+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/3317805491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc101df0 0x7f4bcc1021d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.289+0000 7f4bd17c6640 1 -- 172.21.15.103:0/3317805491 >> 172.21.15.103:0/3317805491 conn(0x7f4bcc0fd8b0 msgr2=0x7f4bcc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.290+0000 7f4bd17c6640 1 -- 172.21.15.103:0/3317805491 shutdown_connections 2024-09-05T02:02:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.290+0000 7f4bd17c6640 1 -- 172.21.15.103:0/3317805491 wait complete. 2024-09-05T02:02:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.290+0000 7f4bd17c6640 1 Processor -- start 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.290+0000 7f4bd17c6640 1 -- start start 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bd17c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc101df0 0x7f4bcc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bd17c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc1027a0 0x7f4bcc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bd17c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106f20 0x7f4bcc19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bd17c6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc111ea0 con 0x7f4bcc106f20 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bd17c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc111d20 con 0x7f4bcc1027a0 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bd17c6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc112020 con 0x7f4bcc101df0 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106f20 0x7f4bcc19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106f20 0x7f4bcc19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58890/0 (socket says 172.21.15.103:58890) 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcb7fe640 1 -- 172.21.15.103:0/1542469139 learned_addr learned my addr 172.21.15.103:0/1542469139 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcaffd640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc101df0 0x7f4bcc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bc27fc640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc1027a0 0x7f4bcc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcb7fe640 1 -- 172.21.15.103:0/1542469139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc101df0 msgr2=0x7f4bcc19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcb7fe640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc101df0 0x7f4bcc19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcb7fe640 1 -- 172.21.15.103:0/1542469139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc1027a0 msgr2=0x7f4bcc19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcb7fe640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc1027a0 0x7f4bcc19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcb7fe640 1 -- 172.21.15.103:0/1542469139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc19f530 con 0x7f4bcc106f20 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bcb7fe640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106f20 0x7f4bcc19ee50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400b970 tx=0x7f4bc400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bc8ff9640 1 -- 172.21.15.103:0/1542469139 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4019070 con 0x7f4bcc106f20 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bc8ff9640 1 -- 172.21.15.103:0/1542469139 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bc4014440 con 0x7f4bcc106f20 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc19f820 con 0x7f4bcc106f20 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bc8ff9640 1 -- 172.21.15.103:0/1542469139 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc40075d0 con 0x7f4bcc106f20 2024-09-05T02:02:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.291+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc1a7100 con 0x7f4bcc106f20 2024-09-05T02:02:38.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.292+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bcc1021d0 con 0x7f4bcc106f20 2024-09-05T02:02:38.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.294+0000 7f4bc8ff9640 1 -- 172.21.15.103:0/1542469139 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4bc401c070 con 0x7f4bcc106f20 2024-09-05T02:02:38.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.294+0000 7f4bc8ff9640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb80a2c00 0x7f4bb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:38.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.294+0000 7f4bcaffd640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb80a2c00 0x7f4bb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:38.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.294+0000 7f4bc8ff9640 1 -- 172.21.15.103:0/1542469139 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4bc40ce930 con 0x7f4bcc106f20 2024-09-05T02:02:38.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.295+0000 7f4bcaffd640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb80a2c00 0x7f4bb80a50c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc005ab0 tx=0x7f4bbc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:38.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.299+0000 7f4bc8ff9640 1 -- 172.21.15.103:0/1542469139 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bc409ae90 con 0x7f4bcc106f20 2024-09-05T02:02:38.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.504+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4bcc1a73a0 con 0x7f4bcc106f20 2024-09-05T02:02:38.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:38 smithi103 ceph-mon[30902]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 614 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:38.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.505+0000 7f4bc8ff9640 1 -- 172.21.15.103:0/1542469139 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f4bc409f190 con 0x7f4bcc106f20 2024-09-05T02:02:38.507 INFO:teuthology.orchestra.run.smithi103.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T02:02:38.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.506+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb80a2c00 msgr2=0x7f4bb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:38.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb80a2c00 0x7f4bb80a50c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc005ab0 tx=0x7f4bbc005a20 comp rx=0 tx=0).stop 2024-09-05T02:02:38.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106f20 msgr2=0x7f4bcc19ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:38.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106f20 0x7f4bcc19ee50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400b970 tx=0x7f4bc400be30 comp rx=0 tx=0).stop 2024-09-05T02:02:38.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 shutdown_connections 2024-09-05T02:02:38.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bcc106f20 0x7f4bcc19ee50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:38.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bcc1027a0 0x7f4bcc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:38.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bb80a2c00 0x7f4bb80a50c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:38.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 --2- 172.21.15.103:0/1542469139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc101df0 0x7f4bcc19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:38.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 >> 172.21.15.103:0/1542469139 conn(0x7f4bcc0fd8b0 msgr2=0x7f4bcc0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:38.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 shutdown_connections 2024-09-05T02:02:38.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:38.507+0000 7f4bd17c6640 1 -- 172.21.15.103:0/1542469139 wait complete. 2024-09-05T02:02:38.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:38 smithi167 ceph-mon[35122]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 614 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:38 smithi139 ceph-mon[34262]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 614 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:39.210 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-05T02:02:39.210 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T02:02:39.210 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-09-05T02:02:39.210 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-05T02:02:39.210 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:39.210 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-05T02:02:39.210 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T02:02:39.211 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T02:02:39.211 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-05T02:02:39.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:39.211 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-09-05T02:02:39.228 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-09-05T02:02:39.229 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T02:02:39.230 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-09-05T02:02:39.230 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T02:02:39.230 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-09-05T02:02:39.230 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-05T02:02:39.230 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42155sec preferred_lft 42155sec 2024-09-05T02:02:39.230 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-05T02:02:39.230 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-05T02:02:39.230 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T02:02:39.230 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:02:39.230 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:02:39.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2024-09-05T02:02:39.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2024-09-05T02:02:39.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2024-09-05T02:02:39.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T02:02:39.231 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:02:39.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:02:39.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:02:39.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:39.372 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T02:02:39.417 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-09-05T02:02:39.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:39.419 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-09-05T02:02:39.498 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-09-05T02:02:39.498 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T02:02:39.499 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-05T02:02:39.499 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-05T02:02:39.499 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T02:02:39.499 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:02:39.499 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:02:39.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-05T02:02:39.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-05T02:02:39.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-05T02:02:39.500 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:02:39.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:02:39.609 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:39 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1542469139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T02:02:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:02:39.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:39 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1542469139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T02:02:39.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:39.640 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-05T02:02:39.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:39 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1542469139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T02:02:39.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:39.658 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-09-05T02:02:39.716 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:02:39.717 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:02:39.717 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T02:02:39.717 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-05T02:02:39.717 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:02:39.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:02:39.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:02:39.834 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-05T02:02:39.835 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:02:39.835 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:02:39.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-05T02:02:39.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T02:02:39.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-05T02:02:39.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-05T02:02:39.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-05T02:02:39.835 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:02:39.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:02:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:02:39.950 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T02:02:39.950 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-05T02:02:39.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2024-09-05T02:02:39.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T02:02:39.951 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-05T02:02:40.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T02:02:40.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:02:40.029 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-05T02:02:40.029 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T02:02:40.029 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:40.045 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-05T02:02:40.045 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T02:02:40.046 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:40.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-09-05T02:02:40.166 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T02:02:40.224 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-09-05T02:02:40.224 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-09-05T02:02:40.224 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-09-05T02:02:40.224 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-09-05T02:02:40.224 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-09-05T02:02:40.224 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-09-05T02:02:40.224 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-09-05T02:02:40.225 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/77f6d94187da0151c1e70b3458f4b0b677f2d2efb39c4b1225614329c7e42f22/merged 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/7fbfcaa433d3550b597826fd248ccdbb900c0b6a9a167e1d7499a35ee0ffb7d4/merged 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-09-05T02:02:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/78a0d766eb04ba5d7d34d49cfc7468281c826fe611704a9cc0f63e424bdf7cb7/merged 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a3796d400e19b2ed58ecb15b7e9b06fe81192e20c2f3a0e084c368230f293d55/merged 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/4de6c1d701ae89ad985fab0fe831f81b6dc5b38046b7f57378a883a517ef3852/merged 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/445a00a99baf26b71312251505aeb4d570d932851a06271e2c149a9151c87b95/merged 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/6179c953998a5ecbb9e302c4e8c2f00517884e41fa1bd24eee12d97bc9f5489a/merged 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a001003b52f9e4db8bc9e1a5c58af8996a4de4eb718a7d35e6bac347b3cd0a96/merged 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/444a0f4c2ba2b20895a50ac0c1043ab2b6ee41bd33a99c8ef5f9d36fbf635330/merged 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/1bb8707282af87813355c77a1093e16db2b83b4475f563a3d35418727c919ad3/merged 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T02:02:40.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:40.228 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-09-05T02:02:40.280 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-05T02:02:40.280 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/326867bd-ec7c-4ddf-a493-aa41608a5e39 --client_fs=cephfs) 2024-09-05T02:02:40.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-09-05T02:02:40.346 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T02:02:40.402 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-09-05T02:02:40.402 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-09-05T02:02:40.402 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-09-05T02:02:40.402 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-09-05T02:02:40.402 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-09-05T02:02:40.403 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/77f6d94187da0151c1e70b3458f4b0b677f2d2efb39c4b1225614329c7e42f22/merged 2024-09-05T02:02:40.404 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/7fbfcaa433d3550b597826fd248ccdbb900c0b6a9a167e1d7499a35ee0ffb7d4/merged 2024-09-05T02:02:40.405 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-09-05T02:02:40.405 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T02:02:40.405 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/78a0d766eb04ba5d7d34d49cfc7468281c826fe611704a9cc0f63e424bdf7cb7/merged 2024-09-05T02:02:40.405 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a3796d400e19b2ed58ecb15b7e9b06fe81192e20c2f3a0e084c368230f293d55/merged 2024-09-05T02:02:40.405 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/4de6c1d701ae89ad985fab0fe831f81b6dc5b38046b7f57378a883a517ef3852/merged 2024-09-05T02:02:40.405 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/445a00a99baf26b71312251505aeb4d570d932851a06271e2c149a9151c87b95/merged 2024-09-05T02:02:40.405 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/6179c953998a5ecbb9e302c4e8c2f00517884e41fa1bd24eee12d97bc9f5489a/merged 2024-09-05T02:02:40.405 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a001003b52f9e4db8bc9e1a5c58af8996a4de4eb718a7d35e6bac347b3cd0a96/merged 2024-09-05T02:02:40.405 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/444a0f4c2ba2b20895a50ac0c1043ab2b6ee41bd33a99c8ef5f9d36fbf635330/merged 2024-09-05T02:02:40.406 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/1bb8707282af87813355c77a1093e16db2b83b4475f563a3d35418727c919ad3/merged 2024-09-05T02:02:40.406 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-09-05T02:02:40.406 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T02:02:40.406 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T02:02:40.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:40.407 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-09-05T02:02:40.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi103.stderr:2024-09-05T02:02:40.403+0000 7f7742069480 -1 init, newargv = 0x563a5701aac0 newargc=15 2024-09-05T02:02:40.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi103.stderr:ceph-fuse[82453]: starting ceph client 2024-09-05T02:02:40.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi103.stderr:ceph-fuse[82453]: starting fuse 2024-09-05T02:02:40.439 INFO:teuthology.orchestra.run.smithi103.stdout:109 2024-09-05T02:02:40.439 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [109] 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo stdin-killer -- python3 -c ' 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> import glob 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> import re 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> import os 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> import subprocess 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> def _find_admin_socket(client_name): 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> files = glob.glob(asok_path) 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-05T02:02:40.440 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> # Given a non-glob path, it better be there 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> if "*" not in asok_path: 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> assert(len(files) == 1) 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> return files[0] 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> for f in files: 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> contents = proc_f.read() 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> if mountpoint in contents: 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> return f 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-05T02:02:40.441 DEBUG:teuthology.orchestra.run.smithi103:> print(_find_admin_socket("client.0")) 2024-09-05T02:02:40.442 DEBUG:teuthology.orchestra.run.smithi103:> ' 2024-09-05T02:02:40.501 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:40 smithi103 ceph-mon[30902]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 515 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:40.558 INFO:teuthology.orchestra.run.smithi103.stdout:/var/run/ceph/ceph-client.0.82453.asok 2024-09-05T02:02:40.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T02:02:40.568 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.82453.asok 2024-09-05T02:02:40.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:40.569 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.82453.asok status 2024-09-05T02:02:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:40 smithi167 ceph-mon[35122]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 515 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:40.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:40 smithi139 ceph-mon[34262]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 515 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:40.702 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:40.702 INFO:teuthology.orchestra.run.smithi103.stdout: "metadata": { 2024-09-05T02:02:40.702 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_sha1": "91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4", 2024-09-05T02:02:40.702 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T02:02:40.702 INFO:teuthology.orchestra.run.smithi103.stdout: "entity_id": "0", 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": "smithi103", 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "pid": "82453", 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "root": "/volumes/qa/sv_0/326867bd-ec7c-4ddf-a493-aa41608a5e39" 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_count": 0, 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_pinned_count": 0, 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 15321, 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "inst": { 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "name": { 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client", 2024-09-05T02:02:40.703 INFO:teuthology.orchestra.run.smithi103.stdout: "num": 15321 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 440282878 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 440282878 2024-09-05T02:02:40.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:02:40.705 INFO:teuthology.orchestra.run.smithi103.stdout: "inst_str": "client.15321 192.168.0.1:0/440282878", 2024-09-05T02:02:40.705 INFO:teuthology.orchestra.run.smithi103.stdout: "addr_str": "192.168.0.1:0/440282878", 2024-09-05T02:02:40.705 INFO:teuthology.orchestra.run.smithi103.stdout: "inode_count": 5, 2024-09-05T02:02:40.705 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_epoch": 28, 2024-09-05T02:02:40.705 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch": 90, 2024-09-05T02:02:40.705 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch_barrier": 0, 2024-09-05T02:02:40.705 INFO:teuthology.orchestra.run.smithi103.stdout: "blocklisted": false, 2024-09-05T02:02:40.705 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs" 2024-09-05T02:02:40.705 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:40.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:40.711 DEBUG:teuthology.orchestra.run.smithi103:> stat --file-system '--printf=%T 2024-09-05T02:02:40.711 DEBUG:teuthology.orchestra.run.smithi103:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:40.727 INFO:teuthology.orchestra.run.smithi103.stdout:fuseblk 2024-09-05T02:02:40.727 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:40.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:02:40.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:40.794 INFO:teuthology.run_tasks:Running task exec... 2024-09-05T02:02:40.805 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T02:02:40.805 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-05T02:02:40.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-05T02:02:40.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.934+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3328606652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c102600 msgr2=0x7fe49c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:40.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.934+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3328606652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c102600 0x7fe49c108e90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe494009ca0 tx=0x7fe49402a8c0 comp rx=0 tx=0).stop 2024-09-05T02:02:40.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.934+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3328606652 shutdown_connections 2024-09-05T02:02:40.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.934+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3328606652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c102600 0x7fe49c108e90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:40.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.934+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3328606652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49c101c40 0x7fe49c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:40.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.934+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3328606652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c100a50 0x7fe49c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:40.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.934+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3328606652 >> 172.21.15.103:0/3328606652 conn(0x7fe49c0fc1b0 msgr2=0x7fe49c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:40.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.934+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3328606652 shutdown_connections 2024-09-05T02:02:40.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.934+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3328606652 wait complete. 2024-09-05T02:02:40.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe4a1fa6640 1 Processor -- start 2024-09-05T02:02:40.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe4a1fa6640 1 -- start start 2024-09-05T02:02:40.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe4a1fa6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c100a50 0x7fe49c197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:40.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe4a1fa6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49c101c40 0x7fe49c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:40.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe4a1fa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c102600 0x7fe49c19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe4a1fa6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10d6f0 con 0x7fe49c100a50 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe4a1fa6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10d570 con 0x7fe49c102600 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe4a1fa6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10d870 con 0x7fe49c101c40 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c100a50 0x7fe49c197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c100a50 0x7fe49c197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55552/0 (socket says 172.21.15.103:55552) 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49b7fe640 1 -- 172.21.15.103:0/3029908324 learned_addr learned my addr 172.21.15.103:0/3029908324 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49affd640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49c101c40 0x7fe49c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49bfff640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c102600 0x7fe49c19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49b7fe640 1 -- 172.21.15.103:0/3029908324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49c101c40 msgr2=0x7fe49c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49b7fe640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49c101c40 0x7fe49c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:40.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49b7fe640 1 -- 172.21.15.103:0/3029908324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c102600 msgr2=0x7fe49c19f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:40.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49b7fe640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c102600 0x7fe49c19f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:40.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49b7fe640 1 -- 172.21.15.103:0/3029908324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c19f910 con 0x7fe49c100a50 2024-09-05T02:02:40.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.935+0000 7fe49affd640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49c101c40 0x7fe49c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:40.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.936+0000 7fe49b7fe640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c100a50 0x7fe49c197d00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c970 tx=0x7fe48c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:40.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.936+0000 7fe498ff9640 1 -- 172.21.15.103:0/3029908324 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c004220 con 0x7fe49c100a50 2024-09-05T02:02:40.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.936+0000 7fe498ff9640 1 -- 172.21.15.103:0/3029908324 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe48c0043c0 con 0x7fe49c100a50 2024-09-05T02:02:40.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.936+0000 7fe498ff9640 1 -- 172.21.15.103:0/3029908324 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c00f450 con 0x7fe49c100a50 2024-09-05T02:02:40.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.936+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c19fc00 con 0x7fe49c100a50 2024-09-05T02:02:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.936+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c1a0150 con 0x7fe49c100a50 2024-09-05T02:02:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.937+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c107df0 con 0x7fe49c100a50 2024-09-05T02:02:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.938+0000 7fe498ff9640 1 -- 172.21.15.103:0/3029908324 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe48c0079b0 con 0x7fe49c100a50 2024-09-05T02:02:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.939+0000 7fe498ff9640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4840a2c30 0x7fe4840a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.939+0000 7fe49affd640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4840a2c30 0x7fe4840a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.939+0000 7fe498ff9640 1 -- 172.21.15.103:0/3029908324 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe48c0cf710 con 0x7fe49c100a50 2024-09-05T02:02:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.939+0000 7fe49affd640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4840a2c30 0x7fe4840a50f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe490009930 tx=0x7fe490008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:40.942+0000 7fe498ff9640 1 -- 172.21.15.103:0/3029908324 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe48c014030 con 0x7fe49c100a50 2024-09-05T02:02:41.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.146+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fe49c02d7c0 con 0x7fe49c100a50 2024-09-05T02:02:41.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.229+0000 7fe498ff9640 1 -- 172.21.15.103:0/3029908324 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7fe48c09c2c0 con 0x7fe49c100a50 2024-09-05T02:02:41.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4840a2c30 msgr2=0x7fe4840a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:41.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4840a2c30 0x7fe4840a50f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe490009930 tx=0x7fe490008040 comp rx=0 tx=0).stop 2024-09-05T02:02:41.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c100a50 msgr2=0x7fe49c197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:41.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c100a50 0x7fe49c197d00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c970 tx=0x7fe48c00ce30 comp rx=0 tx=0).stop 2024-09-05T02:02:41.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 shutdown_connections 2024-09-05T02:02:41.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c102600 0x7fe49c19f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:41.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4840a2c30 0x7fe4840a50f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:41.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49c101c40 0x7fe49c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:41.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 --2- 172.21.15.103:0/3029908324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe49c100a50 0x7fe49c197d00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:41.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 >> 172.21.15.103:0/3029908324 conn(0x7fe49c0fc1b0 msgr2=0x7fe49c104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:41.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 shutdown_connections 2024-09-05T02:02:41.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:41.232+0000 7fe4a1fa6640 1 -- 172.21.15.103:0/3029908324 wait complete. 2024-09-05T02:02:41.244 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-05T02:02:41.257 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-05T02:02:41.280 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-05T02:02:41.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-05T02:02:41.283 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-05T02:02:41.296 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T02:02:41.296 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T02:02:41.546 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:41.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:41 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3029908324' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-05T02:02:41.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:41 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3029908324' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-05T02:02:41.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:41 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3029908324' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-05T02:02:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.033+0000 7f16e4cea640 1 -- 172.21.15.103:0/3622353634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 msgr2=0x7f16e010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.033+0000 7f16e4cea640 1 --2- 172.21.15.103:0/3622353634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 0x7f16e010ea90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009900 tx=0x7f16d002a410 comp rx=0 tx=0).stop 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.033+0000 7f16e4cea640 1 -- 172.21.15.103:0/3622353634 shutdown_connections 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.033+0000 7f16e4cea640 1 --2- 172.21.15.103:0/3622353634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 0x7f16e010ea90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.033+0000 7f16e4cea640 1 --2- 172.21.15.103:0/3622353634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16e0102790 0x7f16e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.033+0000 7f16e4cea640 1 --2- 172.21.15.103:0/3622353634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e0101de0 0x7f16e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.033+0000 7f16e4cea640 1 -- 172.21.15.103:0/3622353634 >> 172.21.15.103:0/3622353634 conn(0x7f16e00fd8c0 msgr2=0x7f16e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.033+0000 7f16e4cea640 1 -- 172.21.15.103:0/3622353634 shutdown_connections 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.033+0000 7f16e4cea640 1 -- 172.21.15.103:0/3622353634 wait complete. 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.034+0000 7f16e4cea640 1 Processor -- start 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.034+0000 7f16e4cea640 1 -- start start 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.034+0000 7f16e4cea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16e0101de0 0x7f16e0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.034+0000 7f16e4cea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e0102790 0x7f16e0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:42.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.034+0000 7f16e4cea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 0x7f16e019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.034+0000 7f16e4cea640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0111e40 con 0x7f16e0106f10 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.034+0000 7f16e4cea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0111cc0 con 0x7f16e0102790 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.034+0000 7f16e4cea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0111fc0 con 0x7f16e0101de0 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ded9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 0x7f16e019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ded9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 0x7f16e019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55580/0 (socket says 172.21.15.103:55580) 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ded9d640 1 -- 172.21.15.103:0/2982510405 learned_addr learned my addr 172.21.15.103:0/2982510405 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ddd9b640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e0102790 0x7f16e0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16de59c640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16e0101de0 0x7f16e0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ded9d640 1 -- 172.21.15.103:0/2982510405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16e0101de0 msgr2=0x7f16e0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ded9d640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16e0101de0 0x7f16e0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ded9d640 1 -- 172.21.15.103:0/2982510405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e0102790 msgr2=0x7f16e0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ded9d640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e0102790 0x7f16e0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ded9d640 1 -- 172.21.15.103:0/2982510405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e019d2d0 con 0x7f16e0106f10 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16ded9d640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 0x7f16e019cbf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009a20 tx=0x7f16d00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16d77fe640 1 -- 172.21.15.103:0/2982510405 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d003f030 con 0x7f16e0106f10 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16d77fe640 1 -- 172.21.15.103:0/2982510405 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d00044d0 con 0x7f16e0106f10 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e019d560 con 0x7f16e0106f10 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16d77fe640 1 -- 172.21.15.103:0/2982510405 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0035070 con 0x7f16e0106f10 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.035+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e01a4ea0 con 0x7f16e0106f10 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.036+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e01021c0 con 0x7f16e0106f10 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.037+0000 7f16d77fe640 1 -- 172.21.15.103:0/2982510405 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f16d0042030 con 0x7f16e0106f10 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.038+0000 7f16d77fe640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.038+0000 7f16de59c640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.038+0000 7f16d77fe640 1 -- 172.21.15.103:0/2982510405 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f16d00ef5b0 con 0x7f16e0106f10 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.039+0000 7f16de59c640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f16d8007350 tx=0x7f16d80072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.042+0000 7f16d77fe640 1 -- 172.21.15.103:0/2982510405 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16d00bc160 con 0x7f16e0106f10 2024-09-05T02:02:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.208+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16e010a160 con 0x7f16cc0a2c00 2024-09-05T02:02:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.218+0000 7f16d77fe640 1 -- 172.21.15.103:0/2982510405 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340533 (secure 0 0 0) 0x7f16d8007e00 con 0x7f16cc0a2c00 2024-09-05T02:02:42.223 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-05T02:02:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f16cc0a2c00 msgr2=0x7f16cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f16d8007350 tx=0x7f16d80072e0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 msgr2=0x7f16e019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 0x7f16e019cbf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009a20 tx=0x7f16d00042e0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 shutdown_connections 2024-09-05T02:02:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0106f10 0x7f16e019cbf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e0102790 0x7f16e0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 --2- 172.21.15.103:0/2982510405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16e0101de0 0x7f16e0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.222+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 >> 172.21.15.103:0/2982510405 conn(0x7f16e00fd8c0 msgr2=0x7f16e00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.223+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 shutdown_connections 2024-09-05T02:02:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:42.223+0000 7f16e4cea640 1 -- 172.21.15.103:0/2982510405 wait complete. 2024-09-05T02:02:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:42 smithi167 ceph-mon[35122]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:42 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3029908324' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-05T02:02:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:42 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:02:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:42 smithi139 ceph-mon[34262]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:42 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3029908324' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-05T02:02:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:42 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:02:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:42 smithi103 ceph-mon[30902]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:42 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3029908324' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-05T02:02:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:42 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:02:42.947 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-05T02:02:41.120243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660417,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1167,"num_read_kb":706,"num_write":441,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392772,"kb_used_data":14336,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549052,"statfs":{"total":1151940427776,"available":1151538229248,"internally_reserved":0,"allocated":14680064,"data_stored":8143409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003387"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:37.371895+0000","last_change":"2024-09-05T02:00:12.347638+0000","last_active":"2024-09-05T02:02:37.371895+0000","last_peered":"2024-09-05T02:02:37.371895+0000","last_clean":"2024-09-05T02:02:37.371895+0000","last_became_active":"2024-09-05T02:00:12.347314+0000","last_became_peered":"2024-09-05T02:00:12.347314+0000","last_unstale":"2024-09-05T02:02:37.371895+0000","last_undegraded":"2024-09-05T02:02:37.371895+0000","last_fullsized":"2024-09-05T02:02:37.371895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:08:39.084117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207767+0000","last_change":"2024-09-05T02:00:14.360262+0000","last_active":"2024-09-05T02:02:27.207767+0000","last_peered":"2024-09-05T02:02:27.207767+0000","last_clean":"2024-09-05T02:02:27.207767+0000","last_became_active":"2024-09-05T02:00:14.359965+0000","last_became_peered":"2024-09-05T02:00:14.359965+0000","last_unstale":"2024-09-05T02:02:27.207767+0000","last_undegraded":"2024-09-05T02:02:27.207767+0000","last_fullsized":"2024-09-05T02:02:27.207767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:01:56.109169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203978+0000","last_change":"2024-09-05T02:00:21.410692+0000","last_active":"2024-09-05T02:02:27.203978+0000","last_peered":"2024-09-05T02:02:27.203978+0000","last_clean":"2024-09-05T02:02:27.203978+0000","last_became_active":"2024-09-05T02:00:21.410298+0000","last_became_peered":"2024-09-05T02:00:21.410298+0000","last_unstale":"2024-09-05T02:02:27.203978+0000","last_undegraded":"2024-09-05T02:02:27.203978+0000","last_fullsized":"2024-09-05T02:02:27.203978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:25:31.241700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207977+0000","last_change":"2024-09-05T02:00:12.783109+0000","last_active":"2024-09-05T02:02:27.207977+0000","last_peered":"2024-09-05T02:02:27.207977+0000","last_clean":"2024-09-05T02:02:27.207977+0000","last_became_active":"2024-09-05T02:00:12.782773+0000","last_became_peered":"2024-09-05T02:00:12.782773+0000","last_unstale":"2024-09-05T02:02:27.207977+0000","last_undegraded":"2024-09-05T02:02:27.207977+0000","last_fullsized":"2024-09-05T02:02:27.207977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:29:12.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200132+0000","last_change":"2024-09-05T02:00:14.370425+0000","last_active":"2024-09-05T02:02:27.200132+0000","last_peered":"2024-09-05T02:02:27.200132+0000","last_clean":"2024-09-05T02:02:27.200132+0000","last_became_active":"2024-09-05T02:00:14.370102+0000","last_became_peered":"2024-09-05T02:00:14.370102+0000","last_unstale":"2024-09-05T02:02:27.200132+0000","last_undegraded":"2024-09-05T02:02:27.200132+0000","last_fullsized":"2024-09-05T02:02:27.200132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:22:43.837418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362081+0000","last_change":"2024-09-05T02:00:21.413082+0000","last_active":"2024-09-05T02:02:27.362081+0000","last_peered":"2024-09-05T02:02:27.362081+0000","last_clean":"2024-09-05T02:02:27.362081+0000","last_became_active":"2024-09-05T02:00:21.412661+0000","last_became_peered":"2024-09-05T02:00:21.412661+0000","last_unstale":"2024-09-05T02:02:27.362081+0000","last_undegraded":"2024-09-05T02:02:27.362081+0000","last_fullsized":"2024-09-05T02:02:27.362081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:51:41.225573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207903+0000","last_change":"2024-09-05T02:00:12.353894+0000","last_active":"2024-09-05T02:02:27.207903+0000","last_peered":"2024-09-05T02:02:27.207903+0000","last_clean":"2024-09-05T02:02:27.207903+0000","last_became_active":"2024-09-05T02:00:12.353483+0000","last_became_peered":"2024-09-05T02:00:12.353483+0000","last_unstale":"2024-09-05T02:02:27.207903+0000","last_undegraded":"2024-09-05T02:02:27.207903+0000","last_fullsized":"2024-09-05T02:02:27.207903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:42.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200374+0000","last_change":"2024-09-05T02:00:14.356441+0000","last_active":"2024-09-05T02:02:27.200374+0000","last_peered":"2024-09-05T02:02:27.200374+0000","last_clean":"2024-09-05T02:02:27.200374+0000","last_became_active":"2024-09-05T02:00:14.356092+0000","last_became_peered":"2024-09-05T02:00:14.356092+0000","last_unstale":"2024-09-05T02:02:27.200374+0000","last_undegraded":"2024-09-05T02:02:27.200374+0000","last_fullsized":"2024-09-05T02:02:27.200374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:33:55.416968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362215+0000","last_change":"2024-09-05T02:00:21.418324+0000","last_active":"2024-09-05T02:02:27.362215+0000","last_peered":"2024-09-05T02:02:27.362215+0000","last_clean":"2024-09-05T02:02:27.362215+0000","last_became_active":"2024-09-05T02:00:21.417934+0000","last_became_peered":"2024-09-05T02:00:21.417934+0000","last_unstale":"2024-09-05T02:02:27.362215+0000","last_undegraded":"2024-09-05T02:02:27.362215+0000","last_fullsized":"2024-09-05T02:02:27.362215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:50.425823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208287+0000","last_change":"2024-09-05T02:00:12.348637+0000","last_active":"2024-09-05T02:02:27.208287+0000","last_peered":"2024-09-05T02:02:27.208287+0000","last_clean":"2024-09-05T02:02:27.208287+0000","last_became_active":"2024-09-05T02:00:12.348287+0000","last_became_peered":"2024-09-05T02:00:12.348287+0000","last_unstale":"2024-09-05T02:02:27.208287+0000","last_undegraded":"2024-09-05T02:02:27.208287+0000","last_fullsized":"2024-09-05T02:02:27.208287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:52:58.368026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207802+0000","last_change":"2024-09-05T02:00:14.368414+0000","last_active":"2024-09-05T02:02:27.207802+0000","last_peered":"2024-09-05T02:02:27.207802+0000","last_clean":"2024-09-05T02:02:27.207802+0000","last_became_active":"2024-09-05T02:00:14.368018+0000","last_became_peered":"2024-09-05T02:00:14.368018+0000","last_unstale":"2024-09-05T02:02:27.207802+0000","last_undegraded":"2024-09-05T02:02:27.207802+0000","last_fullsized":"2024-09-05T02:02:27.207802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:08.246420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361924+0000","last_change":"2024-09-05T02:00:21.416649+0000","last_active":"2024-09-05T02:02:27.361924+0000","last_peered":"2024-09-05T02:02:27.361924+0000","last_clean":"2024-09-05T02:02:27.361924+0000","last_became_active":"2024-09-05T02:00:21.416247+0000","last_became_peered":"2024-09-05T02:00:21.416247+0000","last_unstale":"2024-09-05T02:02:27.361924+0000","last_undegraded":"2024-09-05T02:02:27.361924+0000","last_fullsized":"2024-09-05T02:02:27.361924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:10:00.115655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203731+0000","last_change":"2024-09-05T02:00:14.357719+0000","last_active":"2024-09-05T02:02:27.203731+0000","last_peered":"2024-09-05T02:02:27.203731+0000","last_clean":"2024-09-05T02:02:27.203731+0000","last_became_active":"2024-09-05T02:00:14.357211+0000","last_became_peered":"2024-09-05T02:00:14.357211+0000","last_unstale":"2024-09-05T02:02:27.203731+0000","last_undegraded":"2024-09-05T02:02:27.203731+0000","last_fullsized":"2024-09-05T02:02:27.203731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:27:15.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.214695+0000","last_change":"2024-09-05T02:02:27.214205+0000","last_active":"2024-09-05T02:02:27.214695+0000","last_peered":"2024-09-05T02:02:27.214695+0000","last_clean":"2024-09-05T02:02:27.214695+0000","last_became_active":"2024-09-05T02:02:27.209177+0000","last_became_peered":"2024-09-05T02:02:27.209177+0000","last_unstale":"2024-09-05T02:02:27.214695+0000","last_undegraded":"2024-09-05T02:02:27.214695+0000","last_fullsized":"2024-09-05T02:02:27.214695+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:06:12.009310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362275+0000","last_change":"2024-09-05T02:00:21.415203+0000","last_active":"2024-09-05T02:02:27.362275+0000","last_peered":"2024-09-05T02:02:27.362275+0000","last_clean":"2024-09-05T02:02:27.362275+0000","last_became_active":"2024-09-05T02:00:21.414767+0000","last_became_peered":"2024-09-05T02:00:21.414767+0000","last_unstale":"2024-09-05T02:02:27.362275+0000","last_undegraded":"2024-09-05T02:02:27.362275+0000","last_fullsized":"2024-09-05T02:02:27.362275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:01:48.398448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207759+0000","last_change":"2024-09-05T02:00:14.369417+0000","last_active":"2024-09-05T02:02:27.207759+0000","last_peered":"2024-09-05T02:02:27.207759+0000","last_clean":"2024-09-05T02:02:27.207759+0000","last_became_active":"2024-09-05T02:00:14.369093+0000","last_became_peered":"2024-09-05T02:00:14.369093+0000","last_unstale":"2024-09-05T02:02:27.207759+0000","last_undegraded":"2024-09-05T02:02:27.207759+0000","last_fullsized":"2024-09-05T02:02:27.207759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:01:34.239300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.217832+0000","last_change":"2024-09-05T02:02:27.217395+0000","last_active":"2024-09-05T02:02:27.217832+0000","last_peered":"2024-09-05T02:02:27.217832+0000","last_clean":"2024-09-05T02:02:27.217832+0000","last_became_active":"2024-09-05T02:02:27.210156+0000","last_became_peered":"2024-09-05T02:02:27.210156+0000","last_unstale":"2024-09-05T02:02:27.217832+0000","last_undegraded":"2024-09-05T02:02:27.217832+0000","last_fullsized":"2024-09-05T02:02:27.217832+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:26:02.816593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362455+0000","last_change":"2024-09-05T02:00:21.419070+0000","last_active":"2024-09-05T02:02:27.362455+0000","last_peered":"2024-09-05T02:02:27.362455+0000","last_clean":"2024-09-05T02:02:27.362455+0000","last_became_active":"2024-09-05T02:00:21.418315+0000","last_became_peered":"2024-09-05T02:00:21.418315+0000","last_unstale":"2024-09-05T02:02:27.362455+0000","last_undegraded":"2024-09-05T02:02:27.362455+0000","last_fullsized":"2024-09-05T02:02:27.362455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:17.974047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205618+0000","last_change":"2024-09-05T02:00:14.362965+0000","last_active":"2024-09-05T02:02:27.205618+0000","last_peered":"2024-09-05T02:02:27.205618+0000","last_clean":"2024-09-05T02:02:27.205618+0000","last_became_active":"2024-09-05T02:00:14.361842+0000","last_became_peered":"2024-09-05T02:00:14.361842+0000","last_unstale":"2024-09-05T02:02:27.205618+0000","last_undegraded":"2024-09-05T02:02:27.205618+0000","last_fullsized":"2024-09-05T02:02:27.205618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:45:10.565392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207940+0000","last_change":"2024-09-05T02:00:12.349368+0000","last_active":"2024-09-05T02:02:27.207940+0000","last_peered":"2024-09-05T02:02:27.207940+0000","last_clean":"2024-09-05T02:02:27.207940+0000","last_became_active":"2024-09-05T02:00:12.348990+0000","last_became_peered":"2024-09-05T02:00:12.348990+0000","last_unstale":"2024-09-05T02:02:27.207940+0000","last_undegraded":"2024-09-05T02:02:27.207940+0000","last_fullsized":"2024-09-05T02:02:27.207940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:45:04.998316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358957+0000","last_change":"2024-09-05T02:00:21.415467+0000","last_active":"2024-09-05T02:02:27.358957+0000","last_peered":"2024-09-05T02:02:27.358957+0000","last_clean":"2024-09-05T02:02:27.358957+0000","last_became_active":"2024-09-05T02:00:21.415138+0000","last_became_peered":"2024-09-05T02:00:21.415138+0000","last_unstale":"2024-09-05T02:02:27.358957+0000","last_undegraded":"2024-09-05T02:02:27.358957+0000","last_fullsized":"2024-09-05T02:02:27.358957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:51.221383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207499+0000","last_change":"2024-09-05T02:00:14.357961+0000","last_active":"2024-09-05T02:02:27.207499+0000","last_peered":"2024-09-05T02:02:27.207499+0000","last_clean":"2024-09-05T02:02:27.207499+0000","last_became_active":"2024-09-05T02:00:14.357624+0000","last_became_peered":"2024-09-05T02:00:14.357624+0000","last_unstale":"2024-09-05T02:02:27.207499+0000","last_undegraded":"2024-09-05T02:02:27.207499+0000","last_fullsized":"2024-09-05T02:02:27.207499+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:01.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200268+0000","last_change":"2024-09-05T02:00:12.783304+0000","last_active":"2024-09-05T02:02:27.200268+0000","last_peered":"2024-09-05T02:02:27.200268+0000","last_clean":"2024-09-05T02:02:27.200268+0000","last_became_active":"2024-09-05T02:00:12.782953+0000","last_became_peered":"2024-09-05T02:00:12.782953+0000","last_unstale":"2024-09-05T02:02:27.200268+0000","last_undegraded":"2024-09-05T02:02:27.200268+0000","last_fullsized":"2024-09-05T02:02:27.200268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:50:59.130705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362453+0000","last_change":"2024-09-05T02:00:21.418357+0000","last_active":"2024-09-05T02:02:27.362453+0000","last_peered":"2024-09-05T02:02:27.362453+0000","last_clean":"2024-09-05T02:02:27.362453+0000","last_became_active":"2024-09-05T02:00:21.417779+0000","last_became_peered":"2024-09-05T02:00:21.417779+0000","last_unstale":"2024-09-05T02:02:27.362453+0000","last_undegraded":"2024-09-05T02:02:27.362453+0000","last_fullsized":"2024-09-05T02:02:27.362453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:37:45.423447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200005+0000","last_change":"2024-09-05T02:00:14.362857+0000","last_active":"2024-09-05T02:02:27.200005+0000","last_peered":"2024-09-05T02:02:27.200005+0000","last_clean":"2024-09-05T02:02:27.200005+0000","last_became_active":"2024-09-05T02:00:14.362556+0000","last_became_peered":"2024-09-05T02:00:14.362556+0000","last_unstale":"2024-09-05T02:02:27.200005+0000","last_undegraded":"2024-09-05T02:02:27.200005+0000","last_fullsized":"2024-09-05T02:02:27.200005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:44:03.413055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362826+0000","last_change":"2024-09-05T02:00:12.512747+0000","last_active":"2024-09-05T02:02:27.362826+0000","last_peered":"2024-09-05T02:02:27.362826+0000","last_clean":"2024-09-05T02:02:27.362826+0000","last_became_active":"2024-09-05T02:00:12.512471+0000","last_became_peered":"2024-09-05T02:00:12.512471+0000","last_unstale":"2024-09-05T02:02:27.362826+0000","last_undegraded":"2024-09-05T02:02:27.362826+0000","last_fullsized":"2024-09-05T02:02:27.362826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:50.235020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204153+0000","last_change":"2024-09-05T02:00:21.413947+0000","last_active":"2024-09-05T02:02:27.204153+0000","last_peered":"2024-09-05T02:02:27.204153+0000","last_clean":"2024-09-05T02:02:27.204153+0000","last_became_active":"2024-09-05T02:00:21.413488+0000","last_became_peered":"2024-09-05T02:00:21.413488+0000","last_unstale":"2024-09-05T02:02:27.204153+0000","last_undegraded":"2024-09-05T02:02:27.204153+0000","last_fullsized":"2024-09-05T02:02:27.204153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:52:16.201325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207880+0000","last_change":"2024-09-05T02:00:14.367009+0000","last_active":"2024-09-05T02:02:27.207880+0000","last_peered":"2024-09-05T02:02:27.207880+0000","last_clean":"2024-09-05T02:02:27.207880+0000","last_became_active":"2024-09-05T02:00:14.366698+0000","last_became_peered":"2024-09-05T02:00:14.366698+0000","last_unstale":"2024-09-05T02:02:27.207880+0000","last_undegraded":"2024-09-05T02:02:27.207880+0000","last_fullsized":"2024-09-05T02:02:27.207880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:37:43.811300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'17","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359615+0000","last_change":"2024-09-05T02:00:12.346918+0000","last_active":"2024-09-05T02:02:27.359615+0000","last_peered":"2024-09-05T02:02:27.359615+0000","last_clean":"2024-09-05T02:02:27.359615+0000","last_became_active":"2024-09-05T02:00:12.346541+0000","last_became_peered":"2024-09-05T02:00:12.346541+0000","last_unstale":"2024-09-05T02:02:27.359615+0000","last_undegraded":"2024-09-05T02:02:27.359615+0000","last_fullsized":"2024-09-05T02:02:27.359615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:50.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207371+0000","last_change":"2024-09-05T02:00:21.413077+0000","last_active":"2024-09-05T02:02:27.207371+0000","last_peered":"2024-09-05T02:02:27.207371+0000","last_clean":"2024-09-05T02:02:27.207371+0000","last_became_active":"2024-09-05T02:00:21.412698+0000","last_became_peered":"2024-09-05T02:00:21.412698+0000","last_unstale":"2024-09-05T02:02:27.207371+0000","last_undegraded":"2024-09-05T02:02:27.207371+0000","last_fullsized":"2024-09-05T02:02:27.207371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:12:18.463614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359190+0000","last_change":"2024-09-05T02:00:14.363072+0000","last_active":"2024-09-05T02:02:27.359190+0000","last_peered":"2024-09-05T02:02:27.359190+0000","last_clean":"2024-09-05T02:02:27.359190+0000","last_became_active":"2024-09-05T02:00:14.361932+0000","last_became_peered":"2024-09-05T02:00:14.361932+0000","last_unstale":"2024-09-05T02:02:27.359190+0000","last_undegraded":"2024-09-05T02:02:27.359190+0000","last_fullsized":"2024-09-05T02:02:27.359190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:20:13.080963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208008+0000","last_change":"2024-09-05T02:00:12.783825+0000","last_active":"2024-09-05T02:02:27.208008+0000","last_peered":"2024-09-05T02:02:27.208008+0000","last_clean":"2024-09-05T02:02:27.208008+0000","last_became_active":"2024-09-05T02:00:12.783298+0000","last_became_peered":"2024-09-05T02:00:12.783298+0000","last_unstale":"2024-09-05T02:02:27.208008+0000","last_undegraded":"2024-09-05T02:02:27.208008+0000","last_fullsized":"2024-09-05T02:02:27.208008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:00:10.986318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205534+0000","last_change":"2024-09-05T02:00:21.416474+0000","last_active":"2024-09-05T02:02:27.205534+0000","last_peered":"2024-09-05T02:02:27.205534+0000","last_clean":"2024-09-05T02:02:27.205534+0000","last_became_active":"2024-09-05T02:00:21.416006+0000","last_became_peered":"2024-09-05T02:00:21.416006+0000","last_unstale":"2024-09-05T02:02:27.205534+0000","last_undegraded":"2024-09-05T02:02:27.205534+0000","last_fullsized":"2024-09-05T02:02:27.205534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:00:18.587895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361320+0000","last_change":"2024-09-05T02:00:14.362458+0000","last_active":"2024-09-05T02:02:27.361320+0000","last_peered":"2024-09-05T02:02:27.361320+0000","last_clean":"2024-09-05T02:02:27.361320+0000","last_became_active":"2024-09-05T02:00:14.362080+0000","last_became_peered":"2024-09-05T02:00:14.362080+0000","last_unstale":"2024-09-05T02:02:27.361320+0000","last_undegraded":"2024-09-05T02:02:27.361320+0000","last_fullsized":"2024-09-05T02:02:27.361320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:47:49.498468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359254+0000","last_change":"2024-09-05T02:00:12.783445+0000","last_active":"2024-09-05T02:02:27.359254+0000","last_peered":"2024-09-05T02:02:27.359254+0000","last_clean":"2024-09-05T02:02:27.359254+0000","last_became_active":"2024-09-05T02:00:12.783131+0000","last_became_peered":"2024-09-05T02:00:12.783131+0000","last_unstale":"2024-09-05T02:02:27.359254+0000","last_undegraded":"2024-09-05T02:02:27.359254+0000","last_fullsized":"2024-09-05T02:02:27.359254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:21:57.743629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208964+0000","last_change":"2024-09-05T02:00:21.418568+0000","last_active":"2024-09-05T02:02:27.208964+0000","last_peered":"2024-09-05T02:02:27.208964+0000","last_clean":"2024-09-05T02:02:27.208964+0000","last_became_active":"2024-09-05T02:00:21.418152+0000","last_became_peered":"2024-09-05T02:00:21.418152+0000","last_unstale":"2024-09-05T02:02:27.208964+0000","last_undegraded":"2024-09-05T02:02:27.208964+0000","last_fullsized":"2024-09-05T02:02:27.208964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:50:17.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361500+0000","last_change":"2024-09-05T02:00:14.364556+0000","last_active":"2024-09-05T02:02:27.361500+0000","last_peered":"2024-09-05T02:02:27.361500+0000","last_clean":"2024-09-05T02:02:27.361500+0000","last_became_active":"2024-09-05T02:00:14.364285+0000","last_became_peered":"2024-09-05T02:00:14.364285+0000","last_unstale":"2024-09-05T02:02:27.361500+0000","last_undegraded":"2024-09-05T02:02:27.361500+0000","last_fullsized":"2024-09-05T02:02:27.361500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:24:54.901510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:35.370199+0000","last_change":"2024-09-05T02:00:12.347993+0000","last_active":"2024-09-05T02:02:35.370199+0000","last_peered":"2024-09-05T02:02:35.370199+0000","last_clean":"2024-09-05T02:02:35.370199+0000","last_became_active":"2024-09-05T02:00:12.347656+0000","last_became_peered":"2024-09-05T02:00:12.347656+0000","last_unstale":"2024-09-05T02:02:35.370199+0000","last_undegraded":"2024-09-05T02:02:35.370199+0000","last_fullsized":"2024-09-05T02:02:35.370199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:31:23.820379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358999+0000","last_change":"2024-09-05T02:00:21.410786+0000","last_active":"2024-09-05T02:02:27.358999+0000","last_peered":"2024-09-05T02:02:27.358999+0000","last_clean":"2024-09-05T02:02:27.358999+0000","last_became_active":"2024-09-05T02:00:21.410440+0000","last_became_peered":"2024-09-05T02:00:21.410440+0000","last_unstale":"2024-09-05T02:02:27.358999+0000","last_undegraded":"2024-09-05T02:02:27.358999+0000","last_fullsized":"2024-09-05T02:02:27.358999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:22:14.644413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207789+0000","last_change":"2024-09-05T02:01:26.282147+0000","last_active":"2024-09-05T02:02:27.207789+0000","last_peered":"2024-09-05T02:02:27.207789+0000","last_clean":"2024-09-05T02:02:27.207789+0000","last_became_active":"2024-09-05T02:01:26.281792+0000","last_became_peered":"2024-09-05T02:01:26.281792+0000","last_unstale":"2024-09-05T02:02:27.207789+0000","last_undegraded":"2024-09-05T02:02:27.207789+0000","last_fullsized":"2024-09-05T02:02:27.207789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:10:43.403518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":144,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:36.372621+0000","last_change":"2024-09-05T02:00:12.783358+0000","last_active":"2024-09-05T02:02:36.372621+0000","last_peered":"2024-09-05T02:02:36.372621+0000","last_clean":"2024-09-05T02:02:36.372621+0000","last_became_active":"2024-09-05T02:00:12.782950+0000","last_became_peered":"2024-09-05T02:00:12.782950+0000","last_unstale":"2024-09-05T02:02:36.372621+0000","last_undegraded":"2024-09-05T02:02:36.372621+0000","last_fullsized":"2024-09-05T02:02:36.372621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:17:38.823019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359391+0000","last_change":"2024-09-05T02:00:21.416389+0000","last_active":"2024-09-05T02:02:27.359391+0000","last_peered":"2024-09-05T02:02:27.359391+0000","last_clean":"2024-09-05T02:02:27.359391+0000","last_became_active":"2024-09-05T02:00:21.416065+0000","last_became_peered":"2024-09-05T02:00:21.416065+0000","last_unstale":"2024-09-05T02:02:27.359391+0000","last_undegraded":"2024-09-05T02:02:27.359391+0000","last_fullsized":"2024-09-05T02:02:27.359391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:15.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358605+0000","last_change":"2024-09-05T02:00:14.361404+0000","last_active":"2024-09-05T02:02:27.358605+0000","last_peered":"2024-09-05T02:02:27.358605+0000","last_clean":"2024-09-05T02:02:27.358605+0000","last_became_active":"2024-09-05T02:00:14.361031+0000","last_became_peered":"2024-09-05T02:00:14.361031+0000","last_unstale":"2024-09-05T02:02:27.358605+0000","last_undegraded":"2024-09-05T02:02:27.358605+0000","last_fullsized":"2024-09-05T02:02:27.358605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:40.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.366300+0000","last_change":"2024-09-05T02:02:27.366300+0000","last_active":"2024-09-05T02:02:27.366300+0000","last_peered":"2024-09-05T02:02:27.366300+0000","last_clean":"2024-09-05T02:02:27.366300+0000","last_became_active":"2024-09-05T02:02:27.363278+0000","last_became_peered":"2024-09-05T02:02:27.363278+0000","last_unstale":"2024-09-05T02:02:27.366300+0000","last_undegraded":"2024-09-05T02:02:27.366300+0000","last_fullsized":"2024-09-05T02:02:27.366300+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:13.911137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207760+0000","last_change":"2024-09-05T02:01:26.510839+0000","last_active":"2024-09-05T02:02:27.207760+0000","last_peered":"2024-09-05T02:02:27.207760+0000","last_clean":"2024-09-05T02:02:27.207760+0000","last_became_active":"2024-09-05T02:01:26.509959+0000","last_became_peered":"2024-09-05T02:01:26.509959+0000","last_unstale":"2024-09-05T02:02:27.207760+0000","last_undegraded":"2024-09-05T02:02:27.207760+0000","last_fullsized":"2024-09-05T02:02:27.207760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:57.240190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207721+0000","last_change":"2024-09-05T02:00:14.359356+0000","last_active":"2024-09-05T02:02:27.207721+0000","last_peered":"2024-09-05T02:02:27.207721+0000","last_clean":"2024-09-05T02:02:27.207721+0000","last_became_active":"2024-09-05T02:00:14.359078+0000","last_became_peered":"2024-09-05T02:00:14.359078+0000","last_unstale":"2024-09-05T02:02:27.207721+0000","last_undegraded":"2024-09-05T02:02:27.207721+0000","last_fullsized":"2024-09-05T02:02:27.207721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:40:24.219962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362445+0000","last_change":"2024-09-05T02:00:12.349749+0000","last_active":"2024-09-05T02:02:27.362445+0000","last_peered":"2024-09-05T02:02:27.362445+0000","last_clean":"2024-09-05T02:02:27.362445+0000","last_became_active":"2024-09-05T02:00:12.349247+0000","last_became_peered":"2024-09-05T02:00:12.349247+0000","last_unstale":"2024-09-05T02:02:27.362445+0000","last_undegraded":"2024-09-05T02:02:27.362445+0000","last_fullsized":"2024-09-05T02:02:27.362445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:04:26.361209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203648+0000","last_change":"2024-09-05T01:58:57.552989+0000","last_active":"2024-09-05T02:02:27.203648+0000","last_peered":"2024-09-05T02:02:27.203648+0000","last_clean":"2024-09-05T02:02:27.203648+0000","last_became_active":"2024-09-05T01:58:57.552511+0000","last_became_peered":"2024-09-05T01:58:57.552511+0000","last_unstale":"2024-09-05T02:02:27.203648+0000","last_undegraded":"2024-09-05T02:02:27.203648+0000","last_fullsized":"2024-09-05T02:02:27.203648+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_clean_scrub_stamp":"2024-09-05T01:55:35.678445+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:53.711210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203681+0000","last_change":"2024-09-05T02:00:21.421456+0000","last_active":"2024-09-05T02:02:27.203681+0000","last_peered":"2024-09-05T02:02:27.203681+0000","last_clean":"2024-09-05T02:02:27.203681+0000","last_became_active":"2024-09-05T02:00:21.421074+0000","last_became_peered":"2024-09-05T02:00:21.421074+0000","last_unstale":"2024-09-05T02:02:27.203681+0000","last_undegraded":"2024-09-05T02:02:27.203681+0000","last_fullsized":"2024-09-05T02:02:27.203681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:46:00.660079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204029+0000","last_change":"2024-09-05T02:00:14.357638+0000","last_active":"2024-09-05T02:02:27.204029+0000","last_peered":"2024-09-05T02:02:27.204029+0000","last_clean":"2024-09-05T02:02:27.204029+0000","last_became_active":"2024-09-05T02:00:14.357125+0000","last_became_peered":"2024-09-05T02:00:14.357125+0000","last_unstale":"2024-09-05T02:02:27.204029+0000","last_undegraded":"2024-09-05T02:02:27.204029+0000","last_fullsized":"2024-09-05T02:02:27.204029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:02:04.343235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:38.376320+0000","last_change":"2024-09-05T02:00:12.783627+0000","last_active":"2024-09-05T02:02:38.376320+0000","last_peered":"2024-09-05T02:02:38.376320+0000","last_clean":"2024-09-05T02:02:38.376320+0000","last_became_active":"2024-09-05T02:00:12.783112+0000","last_became_peered":"2024-09-05T02:00:12.783112+0000","last_unstale":"2024-09-05T02:02:38.376320+0000","last_undegraded":"2024-09-05T02:02:38.376320+0000","last_fullsized":"2024-09-05T02:02:38.376320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:39:45.190311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200790+0000","last_change":"2024-09-05T02:00:21.413069+0000","last_active":"2024-09-05T02:02:27.200790+0000","last_peered":"2024-09-05T02:02:27.200790+0000","last_clean":"2024-09-05T02:02:27.200790+0000","last_became_active":"2024-09-05T02:00:21.412687+0000","last_became_peered":"2024-09-05T02:00:21.412687+0000","last_unstale":"2024-09-05T02:02:27.200790+0000","last_undegraded":"2024-09-05T02:02:27.200790+0000","last_fullsized":"2024-09-05T02:02:27.200790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:28.883071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203909+0000","last_change":"2024-09-05T02:00:14.363197+0000","last_active":"2024-09-05T02:02:27.203909+0000","last_peered":"2024-09-05T02:02:27.203909+0000","last_clean":"2024-09-05T02:02:27.203909+0000","last_became_active":"2024-09-05T02:00:14.362854+0000","last_became_peered":"2024-09-05T02:00:14.362854+0000","last_unstale":"2024-09-05T02:02:27.203909+0000","last_undegraded":"2024-09-05T02:02:27.203909+0000","last_fullsized":"2024-09-05T02:02:27.203909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:29:28.770604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208136+0000","last_change":"2024-09-05T02:00:12.348715+0000","last_active":"2024-09-05T02:02:27.208136+0000","last_peered":"2024-09-05T02:02:27.208136+0000","last_clean":"2024-09-05T02:02:27.208136+0000","last_became_active":"2024-09-05T02:00:12.348322+0000","last_became_peered":"2024-09-05T02:00:12.348322+0000","last_unstale":"2024-09-05T02:02:27.208136+0000","last_undegraded":"2024-09-05T02:02:27.208136+0000","last_fullsized":"2024-09-05T02:02:27.208136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:33:55.551080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199905+0000","last_change":"2024-09-05T02:01:26.509548+0000","last_active":"2024-09-05T02:02:27.199905+0000","last_peered":"2024-09-05T02:02:27.199905+0000","last_clean":"2024-09-05T02:02:27.199905+0000","last_became_active":"2024-09-05T02:01:26.508993+0000","last_became_peered":"2024-09-05T02:01:26.508993+0000","last_unstale":"2024-09-05T02:02:27.199905+0000","last_undegraded":"2024-09-05T02:02:27.199905+0000","last_fullsized":"2024-09-05T02:02:27.199905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:46.109685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203973+0000","last_change":"2024-09-05T02:00:14.355969+0000","last_active":"2024-09-05T02:02:27.203973+0000","last_peered":"2024-09-05T02:02:27.203973+0000","last_clean":"2024-09-05T02:02:27.203973+0000","last_became_active":"2024-09-05T02:00:14.355581+0000","last_became_peered":"2024-09-05T02:00:14.355581+0000","last_unstale":"2024-09-05T02:02:27.203973+0000","last_undegraded":"2024-09-05T02:02:27.203973+0000","last_fullsized":"2024-09-05T02:02:27.203973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:55:22.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359490+0000","last_change":"2024-09-05T02:00:12.348148+0000","last_active":"2024-09-05T02:02:27.359490+0000","last_peered":"2024-09-05T02:02:27.359490+0000","last_clean":"2024-09-05T02:02:27.359490+0000","last_became_active":"2024-09-05T02:00:12.347857+0000","last_became_peered":"2024-09-05T02:00:12.347857+0000","last_unstale":"2024-09-05T02:02:27.359490+0000","last_undegraded":"2024-09-05T02:02:27.359490+0000","last_fullsized":"2024-09-05T02:02:27.359490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:22:21.368082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207490+0000","last_change":"2024-09-05T02:00:21.417476+0000","last_active":"2024-09-05T02:02:27.207490+0000","last_peered":"2024-09-05T02:02:27.207490+0000","last_clean":"2024-09-05T02:02:27.207490+0000","last_became_active":"2024-09-05T02:00:21.417036+0000","last_became_peered":"2024-09-05T02:00:21.417036+0000","last_unstale":"2024-09-05T02:02:27.207490+0000","last_undegraded":"2024-09-05T02:02:27.207490+0000","last_fullsized":"2024-09-05T02:02:27.207490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:15:22.510948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205541+0000","last_change":"2024-09-05T02:00:14.358542+0000","last_active":"2024-09-05T02:02:27.205541+0000","last_peered":"2024-09-05T02:02:27.205541+0000","last_clean":"2024-09-05T02:02:27.205541+0000","last_became_active":"2024-09-05T02:00:14.358228+0000","last_became_peered":"2024-09-05T02:00:14.358228+0000","last_unstale":"2024-09-05T02:02:27.205541+0000","last_undegraded":"2024-09-05T02:02:27.205541+0000","last_fullsized":"2024-09-05T02:02:27.205541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:01.627453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208643+0000","last_change":"2024-09-05T02:00:12.345887+0000","last_active":"2024-09-05T02:02:27.208643+0000","last_peered":"2024-09-05T02:02:27.208643+0000","last_clean":"2024-09-05T02:02:27.208643+0000","last_became_active":"2024-09-05T02:00:12.345491+0000","last_became_peered":"2024-09-05T02:00:12.345491+0000","last_unstale":"2024-09-05T02:02:27.208643+0000","last_undegraded":"2024-09-05T02:02:27.208643+0000","last_fullsized":"2024-09-05T02:02:27.208643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:10:32.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207058+0000","last_change":"2024-09-05T02:00:21.418210+0000","last_active":"2024-09-05T02:02:27.207058+0000","last_peered":"2024-09-05T02:02:27.207058+0000","last_clean":"2024-09-05T02:02:27.207058+0000","last_became_active":"2024-09-05T02:00:21.417769+0000","last_became_peered":"2024-09-05T02:00:21.417769+0000","last_unstale":"2024-09-05T02:02:27.207058+0000","last_undegraded":"2024-09-05T02:02:27.207058+0000","last_fullsized":"2024-09-05T02:02:27.207058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:05.050499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207222+0000","last_change":"2024-09-05T02:00:14.367555+0000","last_active":"2024-09-05T02:02:27.207222+0000","last_peered":"2024-09-05T02:02:27.207222+0000","last_clean":"2024-09-05T02:02:27.207222+0000","last_became_active":"2024-09-05T02:00:14.367222+0000","last_became_peered":"2024-09-05T02:00:14.367222+0000","last_unstale":"2024-09-05T02:02:27.207222+0000","last_undegraded":"2024-09-05T02:02:27.207222+0000","last_fullsized":"2024-09-05T02:02:27.207222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:04:26.172141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207700+0000","last_change":"2024-09-05T02:01:26.282875+0000","last_active":"2024-09-05T02:02:27.207700+0000","last_peered":"2024-09-05T02:02:27.207700+0000","last_clean":"2024-09-05T02:02:27.207700+0000","last_became_active":"2024-09-05T02:01:26.271901+0000","last_became_peered":"2024-09-05T02:01:26.271901+0000","last_unstale":"2024-09-05T02:02:27.207700+0000","last_undegraded":"2024-09-05T02:02:27.207700+0000","last_fullsized":"2024-09-05T02:02:27.207700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:29:45.232870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359339+0000","last_change":"2024-09-05T02:00:21.412374+0000","last_active":"2024-09-05T02:02:27.359339+0000","last_peered":"2024-09-05T02:02:27.359339+0000","last_clean":"2024-09-05T02:02:27.359339+0000","last_became_active":"2024-09-05T02:00:21.411926+0000","last_became_peered":"2024-09-05T02:00:21.411926+0000","last_unstale":"2024-09-05T02:02:27.359339+0000","last_undegraded":"2024-09-05T02:02:27.359339+0000","last_fullsized":"2024-09-05T02:02:27.359339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:38:29.657795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203472+0000","last_change":"2024-09-05T02:01:26.269814+0000","last_active":"2024-09-05T02:02:27.203472+0000","last_peered":"2024-09-05T02:02:27.203472+0000","last_clean":"2024-09-05T02:02:27.203472+0000","last_became_active":"2024-09-05T02:01:26.269320+0000","last_became_peered":"2024-09-05T02:01:26.269320+0000","last_unstale":"2024-09-05T02:02:27.203472+0000","last_undegraded":"2024-09-05T02:02:27.203472+0000","last_fullsized":"2024-09-05T02:02:27.203472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:04:21.161986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:37.374155+0000","last_change":"2024-09-05T02:00:12.349175+0000","last_active":"2024-09-05T02:02:37.374155+0000","last_peered":"2024-09-05T02:02:37.374155+0000","last_clean":"2024-09-05T02:02:37.374155+0000","last_became_active":"2024-09-05T02:00:12.348647+0000","last_became_peered":"2024-09-05T02:00:12.348647+0000","last_unstale":"2024-09-05T02:02:37.374155+0000","last_undegraded":"2024-09-05T02:02:37.374155+0000","last_fullsized":"2024-09-05T02:02:37.374155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:38:13.363462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359005+0000","last_change":"2024-09-05T02:00:21.414252+0000","last_active":"2024-09-05T02:02:27.359005+0000","last_peered":"2024-09-05T02:02:27.359005+0000","last_clean":"2024-09-05T02:02:27.359005+0000","last_became_active":"2024-09-05T02:00:21.412483+0000","last_became_peered":"2024-09-05T02:00:21.412483+0000","last_unstale":"2024-09-05T02:02:27.359005+0000","last_undegraded":"2024-09-05T02:02:27.359005+0000","last_fullsized":"2024-09-05T02:02:27.359005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:58.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203867+0000","last_change":"2024-09-05T02:00:14.361018+0000","last_active":"2024-09-05T02:02:27.203867+0000","last_peered":"2024-09-05T02:02:27.203867+0000","last_clean":"2024-09-05T02:02:27.203867+0000","last_became_active":"2024-09-05T02:00:14.360591+0000","last_became_peered":"2024-09-05T02:00:14.360591+0000","last_unstale":"2024-09-05T02:02:27.203867+0000","last_undegraded":"2024-09-05T02:02:27.203867+0000","last_fullsized":"2024-09-05T02:02:27.203867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:49.175681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207571+0000","last_change":"2024-09-05T02:00:12.353257+0000","last_active":"2024-09-05T02:02:27.207571+0000","last_peered":"2024-09-05T02:02:27.207571+0000","last_clean":"2024-09-05T02:02:27.207571+0000","last_became_active":"2024-09-05T02:00:12.352906+0000","last_became_peered":"2024-09-05T02:00:12.352906+0000","last_unstale":"2024-09-05T02:02:27.207571+0000","last_undegraded":"2024-09-05T02:02:27.207571+0000","last_fullsized":"2024-09-05T02:02:27.207571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:53:52.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204956+0000","last_change":"2024-09-05T02:00:21.410346+0000","last_active":"2024-09-05T02:02:27.204956+0000","last_peered":"2024-09-05T02:02:27.204956+0000","last_clean":"2024-09-05T02:02:27.204956+0000","last_became_active":"2024-09-05T02:00:21.409946+0000","last_became_peered":"2024-09-05T02:00:21.409946+0000","last_unstale":"2024-09-05T02:02:27.204956+0000","last_undegraded":"2024-09-05T02:02:27.204956+0000","last_fullsized":"2024-09-05T02:02:27.204956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:56.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358918+0000","last_change":"2024-09-05T02:00:14.360265+0000","last_active":"2024-09-05T02:02:27.358918+0000","last_peered":"2024-09-05T02:02:27.358918+0000","last_clean":"2024-09-05T02:02:27.358918+0000","last_became_active":"2024-09-05T02:00:14.359958+0000","last_became_peered":"2024-09-05T02:00:14.359958+0000","last_unstale":"2024-09-05T02:02:27.358918+0000","last_undegraded":"2024-09-05T02:02:27.358918+0000","last_fullsized":"2024-09-05T02:02:27.358918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:15:45.211207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204648+0000","last_change":"2024-09-05T02:00:12.349443+0000","last_active":"2024-09-05T02:02:27.204648+0000","last_peered":"2024-09-05T02:02:27.204648+0000","last_clean":"2024-09-05T02:02:27.204648+0000","last_became_active":"2024-09-05T02:00:12.348988+0000","last_became_peered":"2024-09-05T02:00:12.348988+0000","last_unstale":"2024-09-05T02:02:27.204648+0000","last_undegraded":"2024-09-05T02:02:27.204648+0000","last_fullsized":"2024-09-05T02:02:27.204648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:14:21.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207693+0000","last_change":"2024-09-05T02:00:21.417662+0000","last_active":"2024-09-05T02:02:27.207693+0000","last_peered":"2024-09-05T02:02:27.207693+0000","last_clean":"2024-09-05T02:02:27.207693+0000","last_became_active":"2024-09-05T02:00:21.417238+0000","last_became_peered":"2024-09-05T02:00:21.417238+0000","last_unstale":"2024-09-05T02:02:27.207693+0000","last_undegraded":"2024-09-05T02:02:27.207693+0000","last_fullsized":"2024-09-05T02:02:27.207693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:05:59.304480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207729+0000","last_change":"2024-09-05T02:00:14.355928+0000","last_active":"2024-09-05T02:02:27.207729+0000","last_peered":"2024-09-05T02:02:27.207729+0000","last_clean":"2024-09-05T02:02:27.207729+0000","last_became_active":"2024-09-05T02:00:14.355576+0000","last_became_peered":"2024-09-05T02:00:14.355576+0000","last_unstale":"2024-09-05T02:02:27.207729+0000","last_undegraded":"2024-09-05T02:02:27.207729+0000","last_fullsized":"2024-09-05T02:02:27.207729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:22:50.095434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'37","reported_seq":286,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:38.375947+0000","last_change":"2024-09-05T02:00:12.352198+0000","last_active":"2024-09-05T02:02:38.375947+0000","last_peered":"2024-09-05T02:02:38.375947+0000","last_clean":"2024-09-05T02:02:38.375947+0000","last_became_active":"2024-09-05T02:00:12.351806+0000","last_became_peered":"2024-09-05T02:00:12.351806+0000","last_unstale":"2024-09-05T02:02:38.375947+0000","last_undegraded":"2024-09-05T02:02:38.375947+0000","last_fullsized":"2024-09-05T02:02:38.375947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.178160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62325,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":210,"num_read_kb":166,"num_write":39,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358666+0000","last_change":"2024-09-05T02:01:26.509575+0000","last_active":"2024-09-05T02:02:27.358666+0000","last_peered":"2024-09-05T02:02:27.358666+0000","last_clean":"2024-09-05T02:02:27.358666+0000","last_became_active":"2024-09-05T02:01:26.509108+0000","last_became_peered":"2024-09-05T02:01:26.509108+0000","last_unstale":"2024-09-05T02:02:27.358666+0000","last_undegraded":"2024-09-05T02:02:27.358666+0000","last_fullsized":"2024-09-05T02:02:27.358666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:55.503181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200586+0000","last_change":"2024-09-05T02:00:14.365372+0000","last_active":"2024-09-05T02:02:27.200586+0000","last_peered":"2024-09-05T02:02:27.200586+0000","last_clean":"2024-09-05T02:02:27.200586+0000","last_became_active":"2024-09-05T02:00:14.365064+0000","last_became_peered":"2024-09-05T02:00:14.365064+0000","last_unstale":"2024-09-05T02:02:27.200586+0000","last_undegraded":"2024-09-05T02:02:27.200586+0000","last_fullsized":"2024-09-05T02:02:27.200586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:02:29.378126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208012+0000","last_change":"2024-09-05T02:00:12.783487+0000","last_active":"2024-09-05T02:02:27.208012+0000","last_peered":"2024-09-05T02:02:27.208012+0000","last_clean":"2024-09-05T02:02:27.208012+0000","last_became_active":"2024-09-05T02:00:12.783087+0000","last_became_peered":"2024-09-05T02:00:12.783087+0000","last_unstale":"2024-09-05T02:02:27.208012+0000","last_undegraded":"2024-09-05T02:02:27.208012+0000","last_fullsized":"2024-09-05T02:02:27.208012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:08.574845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362092+0000","last_change":"2024-09-05T02:01:26.510122+0000","last_active":"2024-09-05T02:02:27.362092+0000","last_peered":"2024-09-05T02:02:27.362092+0000","last_clean":"2024-09-05T02:02:27.362092+0000","last_became_active":"2024-09-05T02:01:26.509427+0000","last_became_peered":"2024-09-05T02:01:26.509427+0000","last_unstale":"2024-09-05T02:02:27.362092+0000","last_undegraded":"2024-09-05T02:02:27.362092+0000","last_fullsized":"2024-09-05T02:02:27.362092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:50.139855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200142+0000","last_change":"2024-09-05T02:01:26.272703+0000","last_active":"2024-09-05T02:02:27.200142+0000","last_peered":"2024-09-05T02:02:27.200142+0000","last_clean":"2024-09-05T02:02:27.200142+0000","last_became_active":"2024-09-05T02:01:26.272440+0000","last_became_peered":"2024-09-05T02:01:26.272440+0000","last_unstale":"2024-09-05T02:02:27.200142+0000","last_undegraded":"2024-09-05T02:02:27.200142+0000","last_fullsized":"2024-09-05T02:02:27.200142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:29.855027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203783+0000","last_change":"2024-09-05T02:00:12.349362+0000","last_active":"2024-09-05T02:02:27.203783+0000","last_peered":"2024-09-05T02:02:27.203783+0000","last_clean":"2024-09-05T02:02:27.203783+0000","last_became_active":"2024-09-05T02:00:12.349016+0000","last_became_peered":"2024-09-05T02:00:12.349016+0000","last_unstale":"2024-09-05T02:02:27.203783+0000","last_undegraded":"2024-09-05T02:02:27.203783+0000","last_fullsized":"2024-09-05T02:02:27.203783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:37.603717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358193+0000","last_change":"2024-09-05T02:00:21.412942+0000","last_active":"2024-09-05T02:02:27.358193+0000","last_peered":"2024-09-05T02:02:27.358193+0000","last_clean":"2024-09-05T02:02:27.358193+0000","last_became_active":"2024-09-05T02:00:21.412057+0000","last_became_peered":"2024-09-05T02:00:21.412057+0000","last_unstale":"2024-09-05T02:02:27.358193+0000","last_undegraded":"2024-09-05T02:02:27.358193+0000","last_fullsized":"2024-09-05T02:02:27.358193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:59.872818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200607+0000","last_change":"2024-09-05T02:00:14.356636+0000","last_active":"2024-09-05T02:02:27.200607+0000","last_peered":"2024-09-05T02:02:27.200607+0000","last_clean":"2024-09-05T02:02:27.200607+0000","last_became_active":"2024-09-05T02:00:14.356321+0000","last_became_peered":"2024-09-05T02:00:14.356321+0000","last_unstale":"2024-09-05T02:02:27.200607+0000","last_undegraded":"2024-09-05T02:02:27.200607+0000","last_fullsized":"2024-09-05T02:02:27.200607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:02.112646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'6","reported_seq":245,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:38.376180+0000","last_change":"2024-09-05T02:00:12.353078+0000","last_active":"2024-09-05T02:02:38.376180+0000","last_peered":"2024-09-05T02:02:38.376180+0000","last_clean":"2024-09-05T02:02:38.376180+0000","last_became_active":"2024-09-05T02:00:12.352707+0000","last_became_peered":"2024-09-05T02:00:12.352707+0000","last_unstale":"2024-09-05T02:02:38.376180+0000","last_undegraded":"2024-09-05T02:02:38.376180+0000","last_fullsized":"2024-09-05T02:02:38.376180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:28:29.934304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207530+0000","last_change":"2024-09-05T02:00:21.421362+0000","last_active":"2024-09-05T02:02:27.207530+0000","last_peered":"2024-09-05T02:02:27.207530+0000","last_clean":"2024-09-05T02:02:27.207530+0000","last_became_active":"2024-09-05T02:00:21.420998+0000","last_became_peered":"2024-09-05T02:00:21.420998+0000","last_unstale":"2024-09-05T02:02:27.207530+0000","last_undegraded":"2024-09-05T02:02:27.207530+0000","last_fullsized":"2024-09-05T02:02:27.207530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:26:14.105489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199804+0000","last_change":"2024-09-05T02:00:21.416703+0000","last_active":"2024-09-05T02:02:27.199804+0000","last_peered":"2024-09-05T02:02:27.199804+0000","last_clean":"2024-09-05T02:02:27.199804+0000","last_became_active":"2024-09-05T02:00:21.415913+0000","last_became_peered":"2024-09-05T02:00:21.415913+0000","last_unstale":"2024-09-05T02:02:27.199804+0000","last_undegraded":"2024-09-05T02:02:27.199804+0000","last_fullsized":"2024-09-05T02:02:27.199804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:56.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358412+0000","last_change":"2024-09-05T02:00:14.355658+0000","last_active":"2024-09-05T02:02:27.358412+0000","last_peered":"2024-09-05T02:02:27.358412+0000","last_clean":"2024-09-05T02:02:27.358412+0000","last_became_active":"2024-09-05T02:00:14.355087+0000","last_became_peered":"2024-09-05T02:00:14.355087+0000","last_unstale":"2024-09-05T02:02:27.358412+0000","last_undegraded":"2024-09-05T02:02:27.358412+0000","last_fullsized":"2024-09-05T02:02:27.358412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:58:08.369027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207984+0000","last_change":"2024-09-05T02:00:12.348171+0000","last_active":"2024-09-05T02:02:27.207984+0000","last_peered":"2024-09-05T02:02:27.207984+0000","last_clean":"2024-09-05T02:02:27.207984+0000","last_became_active":"2024-09-05T02:00:12.347760+0000","last_became_peered":"2024-09-05T02:00:12.347760+0000","last_unstale":"2024-09-05T02:02:27.207984+0000","last_undegraded":"2024-09-05T02:02:27.207984+0000","last_fullsized":"2024-09-05T02:02:27.207984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:03:43.748458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205734+0000","last_change":"2024-09-05T02:00:21.413015+0000","last_active":"2024-09-05T02:02:27.205734+0000","last_peered":"2024-09-05T02:02:27.205734+0000","last_clean":"2024-09-05T02:02:27.205734+0000","last_became_active":"2024-09-05T02:00:21.412631+0000","last_became_peered":"2024-09-05T02:00:21.412631+0000","last_unstale":"2024-09-05T02:02:27.205734+0000","last_undegraded":"2024-09-05T02:02:27.205734+0000","last_fullsized":"2024-09-05T02:02:27.205734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:35.094273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207508+0000","last_change":"2024-09-05T02:00:14.354552+0000","last_active":"2024-09-05T02:02:27.207508+0000","last_peered":"2024-09-05T02:02:27.207508+0000","last_clean":"2024-09-05T02:02:27.207508+0000","last_became_active":"2024-09-05T02:00:14.354150+0000","last_became_peered":"2024-09-05T02:00:14.354150+0000","last_unstale":"2024-09-05T02:02:27.207508+0000","last_undegraded":"2024-09-05T02:02:27.207508+0000","last_fullsized":"2024-09-05T02:02:27.207508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:16:19.836148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358680+0000","last_change":"2024-09-05T02:00:12.342608+0000","last_active":"2024-09-05T02:02:27.358680+0000","last_peered":"2024-09-05T02:02:27.358680+0000","last_clean":"2024-09-05T02:02:27.358680+0000","last_became_active":"2024-09-05T02:00:12.342235+0000","last_became_peered":"2024-09-05T02:00:12.342235+0000","last_unstale":"2024-09-05T02:02:27.358680+0000","last_undegraded":"2024-09-05T02:02:27.358680+0000","last_fullsized":"2024-09-05T02:02:27.358680+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:54.568963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361875+0000","last_change":"2024-09-05T02:00:12.352247+0000","last_active":"2024-09-05T02:02:27.361875+0000","last_peered":"2024-09-05T02:02:27.361875+0000","last_clean":"2024-09-05T02:02:27.361875+0000","last_became_active":"2024-09-05T02:00:12.351897+0000","last_became_peered":"2024-09-05T02:00:12.351897+0000","last_unstale":"2024-09-05T02:02:27.361875+0000","last_undegraded":"2024-09-05T02:02:27.361875+0000","last_fullsized":"2024-09-05T02:02:27.361875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:48.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207250+0000","last_change":"2024-09-05T02:00:14.366415+0000","last_active":"2024-09-05T02:02:27.207250+0000","last_peered":"2024-09-05T02:02:27.207250+0000","last_clean":"2024-09-05T02:02:27.207250+0000","last_became_active":"2024-09-05T02:00:14.366103+0000","last_became_peered":"2024-09-05T02:00:14.366103+0000","last_unstale":"2024-09-05T02:02:27.207250+0000","last_undegraded":"2024-09-05T02:02:27.207250+0000","last_fullsized":"2024-09-05T02:02:27.207250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:22:21.585867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204377+0000","last_change":"2024-09-05T02:00:21.416879+0000","last_active":"2024-09-05T02:02:27.204377+0000","last_peered":"2024-09-05T02:02:27.204377+0000","last_clean":"2024-09-05T02:02:27.204377+0000","last_became_active":"2024-09-05T02:00:21.415887+0000","last_became_peered":"2024-09-05T02:00:21.415887+0000","last_unstale":"2024-09-05T02:02:27.204377+0000","last_undegraded":"2024-09-05T02:02:27.204377+0000","last_fullsized":"2024-09-05T02:02:27.204377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:58.106529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:38.376589+0000","last_change":"2024-09-05T02:00:12.354055+0000","last_active":"2024-09-05T02:02:38.376589+0000","last_peered":"2024-09-05T02:02:38.376589+0000","last_clean":"2024-09-05T02:02:38.376589+0000","last_became_active":"2024-09-05T02:00:12.353754+0000","last_became_peered":"2024-09-05T02:00:12.353754+0000","last_unstale":"2024-09-05T02:02:38.376589+0000","last_undegraded":"2024-09-05T02:02:38.376589+0000","last_fullsized":"2024-09-05T02:02:38.376589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:17.671079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.209049+0000","last_change":"2024-09-05T02:00:14.358646+0000","last_active":"2024-09-05T02:02:27.209049+0000","last_peered":"2024-09-05T02:02:27.209049+0000","last_clean":"2024-09-05T02:02:27.209049+0000","last_became_active":"2024-09-05T02:00:14.358337+0000","last_became_peered":"2024-09-05T02:00:14.358337+0000","last_unstale":"2024-09-05T02:02:27.209049+0000","last_undegraded":"2024-09-05T02:02:27.209049+0000","last_fullsized":"2024-09-05T02:02:27.209049+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:20:39.976731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204237+0000","last_change":"2024-09-05T02:00:21.416611+0000","last_active":"2024-09-05T02:02:27.204237+0000","last_peered":"2024-09-05T02:02:27.204237+0000","last_clean":"2024-09-05T02:02:27.204237+0000","last_became_active":"2024-09-05T02:00:21.416152+0000","last_became_peered":"2024-09-05T02:00:21.416152+0000","last_unstale":"2024-09-05T02:02:27.204237+0000","last_undegraded":"2024-09-05T02:02:27.204237+0000","last_fullsized":"2024-09-05T02:02:27.204237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:20.670383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205477+0000","last_change":"2024-09-05T02:00:14.359015+0000","last_active":"2024-09-05T02:02:27.205477+0000","last_peered":"2024-09-05T02:02:27.205477+0000","last_clean":"2024-09-05T02:02:27.205477+0000","last_became_active":"2024-09-05T02:00:14.358642+0000","last_became_peered":"2024-09-05T02:00:14.358642+0000","last_unstale":"2024-09-05T02:02:27.205477+0000","last_undegraded":"2024-09-05T02:02:27.205477+0000","last_fullsized":"2024-09-05T02:02:27.205477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:18:03.723033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204366+0000","last_change":"2024-09-05T02:00:14.362391+0000","last_active":"2024-09-05T02:02:27.204366+0000","last_peered":"2024-09-05T02:02:27.204366+0000","last_clean":"2024-09-05T02:02:27.204366+0000","last_became_active":"2024-09-05T02:00:14.362007+0000","last_became_peered":"2024-09-05T02:00:14.362007+0000","last_unstale":"2024-09-05T02:02:27.204366+0000","last_undegraded":"2024-09-05T02:02:27.204366+0000","last_fullsized":"2024-09-05T02:02:27.204366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:39:02.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204087+0000","last_change":"2024-09-05T02:01:26.269863+0000","last_active":"2024-09-05T02:02:27.204087+0000","last_peered":"2024-09-05T02:02:27.204087+0000","last_clean":"2024-09-05T02:02:27.204087+0000","last_became_active":"2024-09-05T02:01:26.269451+0000","last_became_peered":"2024-09-05T02:01:26.269451+0000","last_unstale":"2024-09-05T02:02:27.204087+0000","last_undegraded":"2024-09-05T02:02:27.204087+0000","last_fullsized":"2024-09-05T02:02:27.204087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:11.900726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199956+0000","last_change":"2024-09-05T02:00:14.357820+0000","last_active":"2024-09-05T02:02:27.199956+0000","last_peered":"2024-09-05T02:02:27.199956+0000","last_clean":"2024-09-05T02:02:27.199956+0000","last_became_active":"2024-09-05T02:00:14.357320+0000","last_became_peered":"2024-09-05T02:00:14.357320+0000","last_unstale":"2024-09-05T02:02:27.199956+0000","last_undegraded":"2024-09-05T02:02:27.199956+0000","last_fullsized":"2024-09-05T02:02:27.199956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:39.105192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359144+0000","last_change":"2024-09-05T02:00:14.356726+0000","last_active":"2024-09-05T02:02:27.359144+0000","last_peered":"2024-09-05T02:02:27.359144+0000","last_clean":"2024-09-05T02:02:27.359144+0000","last_became_active":"2024-09-05T02:00:14.356452+0000","last_became_peered":"2024-09-05T02:00:14.356452+0000","last_unstale":"2024-09-05T02:02:27.359144+0000","last_undegraded":"2024-09-05T02:02:27.359144+0000","last_fullsized":"2024-09-05T02:02:27.359144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:34:36.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205421+0000","last_change":"2024-09-05T02:00:14.357645+0000","last_active":"2024-09-05T02:02:27.205421+0000","last_peered":"2024-09-05T02:02:27.205421+0000","last_clean":"2024-09-05T02:02:27.205421+0000","last_became_active":"2024-09-05T02:00:14.357330+0000","last_became_peered":"2024-09-05T02:00:14.357330+0000","last_unstale":"2024-09-05T02:02:27.205421+0000","last_undegraded":"2024-09-05T02:02:27.205421+0000","last_fullsized":"2024-09-05T02:02:27.205421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:42.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204836+0000","last_change":"2024-09-05T02:00:14.363413+0000","last_active":"2024-09-05T02:02:27.204836+0000","last_peered":"2024-09-05T02:02:27.204836+0000","last_clean":"2024-09-05T02:02:27.204836+0000","last_became_active":"2024-09-05T02:00:14.362821+0000","last_became_peered":"2024-09-05T02:00:14.362821+0000","last_unstale":"2024-09-05T02:02:27.204836+0000","last_undegraded":"2024-09-05T02:02:27.204836+0000","last_fullsized":"2024-09-05T02:02:27.204836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:49:39.084171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207776+0000","last_change":"2024-09-05T02:00:14.366581+0000","last_active":"2024-09-05T02:02:27.207776+0000","last_peered":"2024-09-05T02:02:27.207776+0000","last_clean":"2024-09-05T02:02:27.207776+0000","last_became_active":"2024-09-05T02:00:14.366287+0000","last_became_peered":"2024-09-05T02:00:14.366287+0000","last_unstale":"2024-09-05T02:02:27.207776+0000","last_undegraded":"2024-09-05T02:02:27.207776+0000","last_fullsized":"2024-09-05T02:02:27.207776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:36:57.143489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207640+0000","last_change":"2024-09-05T02:00:14.360311+0000","last_active":"2024-09-05T02:02:27.207640+0000","last_peered":"2024-09-05T02:02:27.207640+0000","last_clean":"2024-09-05T02:02:27.207640+0000","last_became_active":"2024-09-05T02:00:14.360028+0000","last_became_peered":"2024-09-05T02:00:14.360028+0000","last_unstale":"2024-09-05T02:02:27.207640+0000","last_undegraded":"2024-09-05T02:02:27.207640+0000","last_fullsized":"2024-09-05T02:02:27.207640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:35:38.804555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207867+0000","last_change":"2024-09-05T02:00:14.365853+0000","last_active":"2024-09-05T02:02:27.207867+0000","last_peered":"2024-09-05T02:02:27.207867+0000","last_clean":"2024-09-05T02:02:27.207867+0000","last_became_active":"2024-09-05T02:00:14.365549+0000","last_became_peered":"2024-09-05T02:00:14.365549+0000","last_unstale":"2024-09-05T02:02:27.207867+0000","last_undegraded":"2024-09-05T02:02:27.207867+0000","last_fullsized":"2024-09-05T02:02:27.207867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:58:45.897592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200763+0000","last_change":"2024-09-05T02:00:14.367357+0000","last_active":"2024-09-05T02:02:27.200763+0000","last_peered":"2024-09-05T02:02:27.200763+0000","last_clean":"2024-09-05T02:02:27.200763+0000","last_became_active":"2024-09-05T02:00:14.366794+0000","last_became_peered":"2024-09-05T02:00:14.366794+0000","last_unstale":"2024-09-05T02:02:27.200763+0000","last_undegraded":"2024-09-05T02:02:27.200763+0000","last_fullsized":"2024-09-05T02:02:27.200763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:40:51.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359605+0000","last_change":"2024-09-05T02:00:14.356991+0000","last_active":"2024-09-05T02:02:27.359605+0000","last_peered":"2024-09-05T02:02:27.359605+0000","last_clean":"2024-09-05T02:02:27.359605+0000","last_became_active":"2024-09-05T02:00:14.356662+0000","last_became_peered":"2024-09-05T02:00:14.356662+0000","last_unstale":"2024-09-05T02:02:27.359605+0000","last_undegraded":"2024-09-05T02:02:27.359605+0000","last_fullsized":"2024-09-05T02:02:27.359605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:39:53.087448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205123+0000","last_change":"2024-09-05T02:00:14.357750+0000","last_active":"2024-09-05T02:02:27.205123+0000","last_peered":"2024-09-05T02:02:27.205123+0000","last_clean":"2024-09-05T02:02:27.205123+0000","last_became_active":"2024-09-05T02:00:14.357413+0000","last_became_peered":"2024-09-05T02:00:14.357413+0000","last_unstale":"2024-09-05T02:02:27.205123+0000","last_undegraded":"2024-09-05T02:02:27.205123+0000","last_fullsized":"2024-09-05T02:02:27.205123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:29:47.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204535+0000","last_change":"2024-09-05T02:00:14.364068+0000","last_active":"2024-09-05T02:02:27.204535+0000","last_peered":"2024-09-05T02:02:27.204535+0000","last_clean":"2024-09-05T02:02:27.204535+0000","last_became_active":"2024-09-05T02:00:14.363674+0000","last_became_peered":"2024-09-05T02:00:14.363674+0000","last_unstale":"2024-09-05T02:02:27.204535+0000","last_undegraded":"2024-09-05T02:02:27.204535+0000","last_fullsized":"2024-09-05T02:02:27.204535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:28:00.788668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359585+0000","last_change":"2024-09-05T02:00:14.355152+0000","last_active":"2024-09-05T02:02:27.359585+0000","last_peered":"2024-09-05T02:02:27.359585+0000","last_clean":"2024-09-05T02:02:27.359585+0000","last_became_active":"2024-09-05T02:00:14.354874+0000","last_became_peered":"2024-09-05T02:00:14.354874+0000","last_unstale":"2024-09-05T02:02:27.359585+0000","last_undegraded":"2024-09-05T02:02:27.359585+0000","last_fullsized":"2024-09-05T02:02:27.359585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:14:17.069049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.206078+0000","last_change":"2024-09-05T02:00:14.360241+0000","last_active":"2024-09-05T02:02:27.206078+0000","last_peered":"2024-09-05T02:02:27.206078+0000","last_clean":"2024-09-05T02:02:27.206078+0000","last_became_active":"2024-09-05T02:00:14.359648+0000","last_became_peered":"2024-09-05T02:00:14.359648+0000","last_unstale":"2024-09-05T02:02:27.206078+0000","last_undegraded":"2024-09-05T02:02:27.206078+0000","last_fullsized":"2024-09-05T02:02:27.206078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:15.060575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200191+0000","last_change":"2024-09-05T02:00:14.366094+0000","last_active":"2024-09-05T02:02:27.200191+0000","last_peered":"2024-09-05T02:02:27.200191+0000","last_clean":"2024-09-05T02:02:27.200191+0000","last_became_active":"2024-09-05T02:00:14.365788+0000","last_became_peered":"2024-09-05T02:00:14.365788+0000","last_unstale":"2024-09-05T02:02:27.200191+0000","last_undegraded":"2024-09-05T02:02:27.200191+0000","last_fullsized":"2024-09-05T02:02:27.200191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:23:00.909487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199671+0000","last_change":"2024-09-05T02:00:14.364651+0000","last_active":"2024-09-05T02:02:27.199671+0000","last_peered":"2024-09-05T02:02:27.199671+0000","last_clean":"2024-09-05T02:02:27.199671+0000","last_became_active":"2024-09-05T02:00:14.364365+0000","last_became_peered":"2024-09-05T02:00:14.364365+0000","last_unstale":"2024-09-05T02:02:27.199671+0000","last_undegraded":"2024-09-05T02:02:27.199671+0000","last_fullsized":"2024-09-05T02:02:27.199671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:14.978382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199677+0000","last_change":"2024-09-05T02:00:14.364761+0000","last_active":"2024-09-05T02:02:27.199677+0000","last_peered":"2024-09-05T02:02:27.199677+0000","last_clean":"2024-09-05T02:02:27.199677+0000","last_became_active":"2024-09-05T02:00:14.364204+0000","last_became_peered":"2024-09-05T02:00:14.364204+0000","last_unstale":"2024-09-05T02:02:27.199677+0000","last_undegraded":"2024-09-05T02:02:27.199677+0000","last_fullsized":"2024-09-05T02:02:27.199677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:16:51.076304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359513+0000","last_change":"2024-09-05T02:00:14.357632+0000","last_active":"2024-09-05T02:02:27.359513+0000","last_peered":"2024-09-05T02:02:27.359513+0000","last_clean":"2024-09-05T02:02:27.359513+0000","last_became_active":"2024-09-05T02:00:14.357348+0000","last_became_peered":"2024-09-05T02:00:14.357348+0000","last_unstale":"2024-09-05T02:02:27.359513+0000","last_undegraded":"2024-09-05T02:02:27.359513+0000","last_fullsized":"2024-09-05T02:02:27.359513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:11.163292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362670+0000","last_change":"2024-09-05T02:00:14.361484+0000","last_active":"2024-09-05T02:02:27.362670+0000","last_peered":"2024-09-05T02:02:27.362670+0000","last_clean":"2024-09-05T02:02:27.362670+0000","last_became_active":"2024-09-05T02:00:14.361156+0000","last_became_peered":"2024-09-05T02:00:14.361156+0000","last_unstale":"2024-09-05T02:02:27.362670+0000","last_undegraded":"2024-09-05T02:02:27.362670+0000","last_fullsized":"2024-09-05T02:02:27.362670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:40.124733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203853+0000","last_change":"2024-09-05T02:00:14.358325+0000","last_active":"2024-09-05T02:02:27.203853+0000","last_peered":"2024-09-05T02:02:27.203853+0000","last_clean":"2024-09-05T02:02:27.203853+0000","last_became_active":"2024-09-05T02:00:14.357993+0000","last_became_peered":"2024-09-05T02:00:14.357993+0000","last_unstale":"2024-09-05T02:02:27.203853+0000","last_undegraded":"2024-09-05T02:02:27.203853+0000","last_fullsized":"2024-09-05T02:02:27.203853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:18.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362551+0000","last_change":"2024-09-05T02:00:14.365291+0000","last_active":"2024-09-05T02:02:27.362551+0000","last_peered":"2024-09-05T02:02:27.362551+0000","last_clean":"2024-09-05T02:02:27.362551+0000","last_became_active":"2024-09-05T02:00:14.364937+0000","last_became_peered":"2024-09-05T02:00:14.364937+0000","last_unstale":"2024-09-05T02:02:27.362551+0000","last_undegraded":"2024-09-05T02:02:27.362551+0000","last_fullsized":"2024-09-05T02:02:27.362551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:42:57.712651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200748+0000","last_change":"2024-09-05T02:00:14.367603+0000","last_active":"2024-09-05T02:02:27.200748+0000","last_peered":"2024-09-05T02:02:27.200748+0000","last_clean":"2024-09-05T02:02:27.200748+0000","last_became_active":"2024-09-05T02:00:14.367267+0000","last_became_peered":"2024-09-05T02:00:14.367267+0000","last_unstale":"2024-09-05T02:02:27.200748+0000","last_undegraded":"2024-09-05T02:02:27.200748+0000","last_fullsized":"2024-09-05T02:02:27.200748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:51:34.595898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207492+0000","last_change":"2024-09-05T02:00:14.369499+0000","last_active":"2024-09-05T02:02:27.207492+0000","last_peered":"2024-09-05T02:02:27.207492+0000","last_clean":"2024-09-05T02:02:27.207492+0000","last_became_active":"2024-09-05T02:00:14.369169+0000","last_became_peered":"2024-09-05T02:00:14.369169+0000","last_unstale":"2024-09-05T02:02:27.207492+0000","last_undegraded":"2024-09-05T02:02:27.207492+0000","last_fullsized":"2024-09-05T02:02:27.207492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:17.161204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203441+0000","last_change":"2024-09-05T02:00:14.360320+0000","last_active":"2024-09-05T02:02:27.203441+0000","last_peered":"2024-09-05T02:02:27.203441+0000","last_clean":"2024-09-05T02:02:27.203441+0000","last_became_active":"2024-09-05T02:00:14.359991+0000","last_became_peered":"2024-09-05T02:00:14.359991+0000","last_unstale":"2024-09-05T02:02:27.203441+0000","last_undegraded":"2024-09-05T02:02:27.203441+0000","last_fullsized":"2024-09-05T02:02:27.203441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:03.721863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361323+0000","last_change":"2024-09-05T02:00:14.364660+0000","last_active":"2024-09-05T02:02:27.361323+0000","last_peered":"2024-09-05T02:02:27.361323+0000","last_clean":"2024-09-05T02:02:27.361323+0000","last_became_active":"2024-09-05T02:00:14.364313+0000","last_became_peered":"2024-09-05T02:00:14.364313+0000","last_unstale":"2024-09-05T02:02:27.361323+0000","last_undegraded":"2024-09-05T02:02:27.361323+0000","last_fullsized":"2024-09-05T02:02:27.361323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:47.496063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208875+0000","last_change":"2024-09-05T02:00:14.359853+0000","last_active":"2024-09-05T02:02:27.208875+0000","last_peered":"2024-09-05T02:02:27.208875+0000","last_clean":"2024-09-05T02:02:27.208875+0000","last_became_active":"2024-09-05T02:00:14.359543+0000","last_became_peered":"2024-09-05T02:00:14.359543+0000","last_unstale":"2024-09-05T02:02:27.208875+0000","last_undegraded":"2024-09-05T02:02:27.208875+0000","last_fullsized":"2024-09-05T02:02:27.208875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:30:29.415492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358876+0000","last_change":"2024-09-05T02:01:26.270888+0000","last_active":"2024-09-05T02:02:27.358876+0000","last_peered":"2024-09-05T02:02:27.358876+0000","last_clean":"2024-09-05T02:02:27.358876+0000","last_became_active":"2024-09-05T02:01:26.270607+0000","last_became_peered":"2024-09-05T02:01:26.270607+0000","last_unstale":"2024-09-05T02:02:27.358876+0000","last_undegraded":"2024-09-05T02:02:27.358876+0000","last_fullsized":"2024-09-05T02:02:27.358876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:16.563521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199910+0000","last_change":"2024-09-05T02:00:14.367191+0000","last_active":"2024-09-05T02:02:27.199910+0000","last_peered":"2024-09-05T02:02:27.199910+0000","last_clean":"2024-09-05T02:02:27.199910+0000","last_became_active":"2024-09-05T02:00:14.366831+0000","last_became_peered":"2024-09-05T02:00:14.366831+0000","last_unstale":"2024-09-05T02:02:27.199910+0000","last_undegraded":"2024-09-05T02:02:27.199910+0000","last_fullsized":"2024-09-05T02:02:27.199910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:55.690642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199853+0000","last_change":"2024-09-05T02:00:14.355774+0000","last_active":"2024-09-05T02:02:27.199853+0000","last_peered":"2024-09-05T02:02:27.199853+0000","last_clean":"2024-09-05T02:02:27.199853+0000","last_became_active":"2024-09-05T02:00:14.355293+0000","last_became_peered":"2024-09-05T02:00:14.355293+0000","last_unstale":"2024-09-05T02:02:27.199853+0000","last_undegraded":"2024-09-05T02:02:27.199853+0000","last_fullsized":"2024-09-05T02:02:27.199853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:07:00.547820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205824+0000","last_change":"2024-09-05T02:00:14.359978+0000","last_active":"2024-09-05T02:02:27.205824+0000","last_peered":"2024-09-05T02:02:27.205824+0000","last_clean":"2024-09-05T02:02:27.205824+0000","last_became_active":"2024-09-05T02:00:14.359209+0000","last_became_peered":"2024-09-05T02:00:14.359209+0000","last_unstale":"2024-09-05T02:02:27.205824+0000","last_undegraded":"2024-09-05T02:02:27.205824+0000","last_fullsized":"2024-09-05T02:02:27.205824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:27.848906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69815,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1041,"num_read_kb":597,"num_write":258,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":209445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":516663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1736704,"data_stored":1165270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1069056,"data_stored":520155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201339,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":516437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1658880,"data_stored":1113077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593797,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":511563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822729,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36980,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708172,"statfs":{"total":95995035648,"available":95957168128,"internally_reserved":0,"allocated":1667072,"data_stored":1108659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986255,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":515749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1052672,"data_stored":511271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444120,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1122304,"data_stored":576094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1110016,"data_stored":574918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1064960,"data_stored":513553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T02:02:42.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T02:02:43.199 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:43.494 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:43 smithi103 ceph-mon[30902]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:02:43.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:43 smithi167 ceph-mon[35122]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:02:43.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:43 smithi139 ceph-mon[34262]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:02:43.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.667+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1644717722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 msgr2=0x7fd2b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.667+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1644717722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 0x7fd2b01021c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009900 tx=0x7fd2a002a410 comp rx=0 tx=0).stop 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.667+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1644717722 shutdown_connections 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.667+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1644717722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2b0106f10 0x7fd2b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.667+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1644717722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2b0102790 0x7fd2b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.667+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1644717722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 0x7fd2b01021c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.667+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1644717722 >> 172.21.15.103:0/1644717722 conn(0x7fd2b00fd8c0 msgr2=0x7fd2b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.668+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1644717722 shutdown_connections 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.668+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1644717722 wait complete. 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.668+0000 7fd2b4ea8640 1 Processor -- start 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2b4ea8640 1 -- start start 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2b4ea8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 0x7fd2b01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2b4ea8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2b0102790 0x7fd2b0198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2b4ea8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2b0106f10 0x7fd2b019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2b4ea8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b0111e40 con 0x7fd2b0101de0 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2b4ea8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b0111cc0 con 0x7fd2b0102790 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2b4ea8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b0111fc0 con 0x7fd2b0106f10 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 0x7fd2b01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 0x7fd2b01983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55594/0 (socket says 172.21.15.103:55594) 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2ae59c640 1 -- 172.21.15.103:0/1445721600 learned_addr learned my addr 172.21.15.103:0/1445721600 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2aed9d640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2b0106f10 0x7fd2b019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2ae59c640 1 -- 172.21.15.103:0/1445721600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2b0106f10 msgr2=0x7fd2b019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2ae59c640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2b0106f10 0x7fd2b019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2ae59c640 1 -- 172.21.15.103:0/1445721600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2b0102790 msgr2=0x7fd2b0198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2ae59c640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2b0102790 0x7fd2b0198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2ae59c640 1 -- 172.21.15.103:0/1445721600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b019d390 con 0x7fd2b0101de0 2024-09-05T02:02:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd2ae59c640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 0x7fd2b01983e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0006010 tx=0x7fd2a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.669+0000 7fd29f7fe640 1 -- 172.21.15.103:0/1445721600 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a003f030 con 0x7fd2b0101de0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.670+0000 7fd29f7fe640 1 -- 172.21.15.103:0/1445721600 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2a00035f0 con 0x7fd2b0101de0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.670+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b019d620 con 0x7fd2b0101de0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.670+0000 7fd29f7fe640 1 -- 172.21.15.103:0/1445721600 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0035070 con 0x7fd2b0101de0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.670+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b01a4f60 con 0x7fd2b0101de0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.671+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b01032c0 con 0x7fd2b0101de0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.672+0000 7fd29f7fe640 1 -- 172.21.15.103:0/1445721600 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd2a0003b10 con 0x7fd2b0101de0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.673+0000 7fd29f7fe640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2980a2b80 0x7fd2980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.673+0000 7fd2add9b640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2980a2b80 0x7fd2980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.673+0000 7fd29f7fe640 1 -- 172.21.15.103:0/1445721600 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd2a00f0230 con 0x7fd2b0101de0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.674+0000 7fd2add9b640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2980a2b80 0x7fd2980a5040 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00ffde0 tx=0x7fd2a4005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:43.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.676+0000 7fd29f7fe640 1 -- 172.21.15.103:0/1445721600 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2a00bcde0 con 0x7fd2b0101de0 2024-09-05T02:02:43.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.838+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2b0062630 con 0x7fd2980a2b80 2024-09-05T02:02:43.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.846+0000 7fd29f7fe640 1 -- 172.21.15.103:0/1445721600 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340535 (secure 0 0 0) 0x7fd2b0062630 con 0x7fd2980a2b80 2024-09-05T02:02:43.851 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:43.852 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2980a2b80 msgr2=0x7fd2980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2980a2b80 0x7fd2980a5040 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00ffde0 tx=0x7fd2a4005bf0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 msgr2=0x7fd2b01983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 0x7fd2b01983e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0006010 tx=0x7fd2a0004270 comp rx=0 tx=0).stop 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 shutdown_connections 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2980a2b80 0x7fd2980a5040 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2b0106f10 0x7fd2b019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2b0102790 0x7fd2b0198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 --2- 172.21.15.103:0/1445721600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2b0101de0 0x7fd2b01983e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:43.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 >> 172.21.15.103:0/1445721600 conn(0x7fd2b00fd8c0 msgr2=0x7fd2b00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:43.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 shutdown_connections 2024-09-05T02:02:43.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:43.849+0000 7fd2b4ea8640 1 -- 172.21.15.103:0/1445721600 wait complete. 2024-09-05T02:02:44.587 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:44 smithi103 ceph-mon[30902]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:44.621 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-05T02:02:43.120728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660912,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1173,"num_read_kb":707,"num_write":442,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392760,"kb_used_data":14336,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549064,"statfs":{"total":1151940427776,"available":1151538241536,"internally_reserved":0,"allocated":14680064,"data_stored":8146379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":495,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":51,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003169"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:37.371895+0000","last_change":"2024-09-05T02:00:12.347638+0000","last_active":"2024-09-05T02:02:37.371895+0000","last_peered":"2024-09-05T02:02:37.371895+0000","last_clean":"2024-09-05T02:02:37.371895+0000","last_became_active":"2024-09-05T02:00:12.347314+0000","last_became_peered":"2024-09-05T02:00:12.347314+0000","last_unstale":"2024-09-05T02:02:37.371895+0000","last_undegraded":"2024-09-05T02:02:37.371895+0000","last_fullsized":"2024-09-05T02:02:37.371895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:08:39.084117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207767+0000","last_change":"2024-09-05T02:00:14.360262+0000","last_active":"2024-09-05T02:02:27.207767+0000","last_peered":"2024-09-05T02:02:27.207767+0000","last_clean":"2024-09-05T02:02:27.207767+0000","last_became_active":"2024-09-05T02:00:14.359965+0000","last_became_peered":"2024-09-05T02:00:14.359965+0000","last_unstale":"2024-09-05T02:02:27.207767+0000","last_undegraded":"2024-09-05T02:02:27.207767+0000","last_fullsized":"2024-09-05T02:02:27.207767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:01:56.109169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203978+0000","last_change":"2024-09-05T02:00:21.410692+0000","last_active":"2024-09-05T02:02:27.203978+0000","last_peered":"2024-09-05T02:02:27.203978+0000","last_clean":"2024-09-05T02:02:27.203978+0000","last_became_active":"2024-09-05T02:00:21.410298+0000","last_became_peered":"2024-09-05T02:00:21.410298+0000","last_unstale":"2024-09-05T02:02:27.203978+0000","last_undegraded":"2024-09-05T02:02:27.203978+0000","last_fullsized":"2024-09-05T02:02:27.203978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:25:31.241700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207977+0000","last_change":"2024-09-05T02:00:12.783109+0000","last_active":"2024-09-05T02:02:27.207977+0000","last_peered":"2024-09-05T02:02:27.207977+0000","last_clean":"2024-09-05T02:02:27.207977+0000","last_became_active":"2024-09-05T02:00:12.782773+0000","last_became_peered":"2024-09-05T02:00:12.782773+0000","last_unstale":"2024-09-05T02:02:27.207977+0000","last_undegraded":"2024-09-05T02:02:27.207977+0000","last_fullsized":"2024-09-05T02:02:27.207977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:29:12.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200132+0000","last_change":"2024-09-05T02:00:14.370425+0000","last_active":"2024-09-05T02:02:27.200132+0000","last_peered":"2024-09-05T02:02:27.200132+0000","last_clean":"2024-09-05T02:02:27.200132+0000","last_became_active":"2024-09-05T02:00:14.370102+0000","last_became_peered":"2024-09-05T02:00:14.370102+0000","last_unstale":"2024-09-05T02:02:27.200132+0000","last_undegraded":"2024-09-05T02:02:27.200132+0000","last_fullsized":"2024-09-05T02:02:27.200132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:22:43.837418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362081+0000","last_change":"2024-09-05T02:00:21.413082+0000","last_active":"2024-09-05T02:02:27.362081+0000","last_peered":"2024-09-05T02:02:27.362081+0000","last_clean":"2024-09-05T02:02:27.362081+0000","last_became_active":"2024-09-05T02:00:21.412661+0000","last_became_peered":"2024-09-05T02:00:21.412661+0000","last_unstale":"2024-09-05T02:02:27.362081+0000","last_undegraded":"2024-09-05T02:02:27.362081+0000","last_fullsized":"2024-09-05T02:02:27.362081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:51:41.225573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207903+0000","last_change":"2024-09-05T02:00:12.353894+0000","last_active":"2024-09-05T02:02:27.207903+0000","last_peered":"2024-09-05T02:02:27.207903+0000","last_clean":"2024-09-05T02:02:27.207903+0000","last_became_active":"2024-09-05T02:00:12.353483+0000","last_became_peered":"2024-09-05T02:00:12.353483+0000","last_unstale":"2024-09-05T02:02:27.207903+0000","last_undegraded":"2024-09-05T02:02:27.207903+0000","last_fullsized":"2024-09-05T02:02:27.207903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:42.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200374+0000","last_change":"2024-09-05T02:00:14.356441+0000","last_active":"2024-09-05T02:02:27.200374+0000","last_peered":"2024-09-05T02:02:27.200374+0000","last_clean":"2024-09-05T02:02:27.200374+0000","last_became_active":"2024-09-05T02:00:14.356092+0000","last_became_peered":"2024-09-05T02:00:14.356092+0000","last_unstale":"2024-09-05T02:02:27.200374+0000","last_undegraded":"2024-09-05T02:02:27.200374+0000","last_fullsized":"2024-09-05T02:02:27.200374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:33:55.416968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362215+0000","last_change":"2024-09-05T02:00:21.418324+0000","last_active":"2024-09-05T02:02:27.362215+0000","last_peered":"2024-09-05T02:02:27.362215+0000","last_clean":"2024-09-05T02:02:27.362215+0000","last_became_active":"2024-09-05T02:00:21.417934+0000","last_became_peered":"2024-09-05T02:00:21.417934+0000","last_unstale":"2024-09-05T02:02:27.362215+0000","last_undegraded":"2024-09-05T02:02:27.362215+0000","last_fullsized":"2024-09-05T02:02:27.362215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:50.425823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208287+0000","last_change":"2024-09-05T02:00:12.348637+0000","last_active":"2024-09-05T02:02:27.208287+0000","last_peered":"2024-09-05T02:02:27.208287+0000","last_clean":"2024-09-05T02:02:27.208287+0000","last_became_active":"2024-09-05T02:00:12.348287+0000","last_became_peered":"2024-09-05T02:00:12.348287+0000","last_unstale":"2024-09-05T02:02:27.208287+0000","last_undegraded":"2024-09-05T02:02:27.208287+0000","last_fullsized":"2024-09-05T02:02:27.208287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:52:58.368026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207802+0000","last_change":"2024-09-05T02:00:14.368414+0000","last_active":"2024-09-05T02:02:27.207802+0000","last_peered":"2024-09-05T02:02:27.207802+0000","last_clean":"2024-09-05T02:02:27.207802+0000","last_became_active":"2024-09-05T02:00:14.368018+0000","last_became_peered":"2024-09-05T02:00:14.368018+0000","last_unstale":"2024-09-05T02:02:27.207802+0000","last_undegraded":"2024-09-05T02:02:27.207802+0000","last_fullsized":"2024-09-05T02:02:27.207802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:08.246420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361924+0000","last_change":"2024-09-05T02:00:21.416649+0000","last_active":"2024-09-05T02:02:27.361924+0000","last_peered":"2024-09-05T02:02:27.361924+0000","last_clean":"2024-09-05T02:02:27.361924+0000","last_became_active":"2024-09-05T02:00:21.416247+0000","last_became_peered":"2024-09-05T02:00:21.416247+0000","last_unstale":"2024-09-05T02:02:27.361924+0000","last_undegraded":"2024-09-05T02:02:27.361924+0000","last_fullsized":"2024-09-05T02:02:27.361924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:10:00.115655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203731+0000","last_change":"2024-09-05T02:00:14.357719+0000","last_active":"2024-09-05T02:02:27.203731+0000","last_peered":"2024-09-05T02:02:27.203731+0000","last_clean":"2024-09-05T02:02:27.203731+0000","last_became_active":"2024-09-05T02:00:14.357211+0000","last_became_peered":"2024-09-05T02:00:14.357211+0000","last_unstale":"2024-09-05T02:02:27.203731+0000","last_undegraded":"2024-09-05T02:02:27.203731+0000","last_fullsized":"2024-09-05T02:02:27.203731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:27:15.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.214695+0000","last_change":"2024-09-05T02:02:27.214205+0000","last_active":"2024-09-05T02:02:27.214695+0000","last_peered":"2024-09-05T02:02:27.214695+0000","last_clean":"2024-09-05T02:02:27.214695+0000","last_became_active":"2024-09-05T02:02:27.209177+0000","last_became_peered":"2024-09-05T02:02:27.209177+0000","last_unstale":"2024-09-05T02:02:27.214695+0000","last_undegraded":"2024-09-05T02:02:27.214695+0000","last_fullsized":"2024-09-05T02:02:27.214695+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:06:12.009310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362275+0000","last_change":"2024-09-05T02:00:21.415203+0000","last_active":"2024-09-05T02:02:27.362275+0000","last_peered":"2024-09-05T02:02:27.362275+0000","last_clean":"2024-09-05T02:02:27.362275+0000","last_became_active":"2024-09-05T02:00:21.414767+0000","last_became_peered":"2024-09-05T02:00:21.414767+0000","last_unstale":"2024-09-05T02:02:27.362275+0000","last_undegraded":"2024-09-05T02:02:27.362275+0000","last_fullsized":"2024-09-05T02:02:27.362275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:01:48.398448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207759+0000","last_change":"2024-09-05T02:00:14.369417+0000","last_active":"2024-09-05T02:02:27.207759+0000","last_peered":"2024-09-05T02:02:27.207759+0000","last_clean":"2024-09-05T02:02:27.207759+0000","last_became_active":"2024-09-05T02:00:14.369093+0000","last_became_peered":"2024-09-05T02:00:14.369093+0000","last_unstale":"2024-09-05T02:02:27.207759+0000","last_undegraded":"2024-09-05T02:02:27.207759+0000","last_fullsized":"2024-09-05T02:02:27.207759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:01:34.239300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.217832+0000","last_change":"2024-09-05T02:02:27.217395+0000","last_active":"2024-09-05T02:02:27.217832+0000","last_peered":"2024-09-05T02:02:27.217832+0000","last_clean":"2024-09-05T02:02:27.217832+0000","last_became_active":"2024-09-05T02:02:27.210156+0000","last_became_peered":"2024-09-05T02:02:27.210156+0000","last_unstale":"2024-09-05T02:02:27.217832+0000","last_undegraded":"2024-09-05T02:02:27.217832+0000","last_fullsized":"2024-09-05T02:02:27.217832+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:26:02.816593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362455+0000","last_change":"2024-09-05T02:00:21.419070+0000","last_active":"2024-09-05T02:02:27.362455+0000","last_peered":"2024-09-05T02:02:27.362455+0000","last_clean":"2024-09-05T02:02:27.362455+0000","last_became_active":"2024-09-05T02:00:21.418315+0000","last_became_peered":"2024-09-05T02:00:21.418315+0000","last_unstale":"2024-09-05T02:02:27.362455+0000","last_undegraded":"2024-09-05T02:02:27.362455+0000","last_fullsized":"2024-09-05T02:02:27.362455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:17.974047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205618+0000","last_change":"2024-09-05T02:00:14.362965+0000","last_active":"2024-09-05T02:02:27.205618+0000","last_peered":"2024-09-05T02:02:27.205618+0000","last_clean":"2024-09-05T02:02:27.205618+0000","last_became_active":"2024-09-05T02:00:14.361842+0000","last_became_peered":"2024-09-05T02:00:14.361842+0000","last_unstale":"2024-09-05T02:02:27.205618+0000","last_undegraded":"2024-09-05T02:02:27.205618+0000","last_fullsized":"2024-09-05T02:02:27.205618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:45:10.565392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207940+0000","last_change":"2024-09-05T02:00:12.349368+0000","last_active":"2024-09-05T02:02:27.207940+0000","last_peered":"2024-09-05T02:02:27.207940+0000","last_clean":"2024-09-05T02:02:27.207940+0000","last_became_active":"2024-09-05T02:00:12.348990+0000","last_became_peered":"2024-09-05T02:00:12.348990+0000","last_unstale":"2024-09-05T02:02:27.207940+0000","last_undegraded":"2024-09-05T02:02:27.207940+0000","last_fullsized":"2024-09-05T02:02:27.207940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:45:04.998316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358957+0000","last_change":"2024-09-05T02:00:21.415467+0000","last_active":"2024-09-05T02:02:27.358957+0000","last_peered":"2024-09-05T02:02:27.358957+0000","last_clean":"2024-09-05T02:02:27.358957+0000","last_became_active":"2024-09-05T02:00:21.415138+0000","last_became_peered":"2024-09-05T02:00:21.415138+0000","last_unstale":"2024-09-05T02:02:27.358957+0000","last_undegraded":"2024-09-05T02:02:27.358957+0000","last_fullsized":"2024-09-05T02:02:27.358957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:51.221383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207499+0000","last_change":"2024-09-05T02:00:14.357961+0000","last_active":"2024-09-05T02:02:27.207499+0000","last_peered":"2024-09-05T02:02:27.207499+0000","last_clean":"2024-09-05T02:02:27.207499+0000","last_became_active":"2024-09-05T02:00:14.357624+0000","last_became_peered":"2024-09-05T02:00:14.357624+0000","last_unstale":"2024-09-05T02:02:27.207499+0000","last_undegraded":"2024-09-05T02:02:27.207499+0000","last_fullsized":"2024-09-05T02:02:27.207499+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:01.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200268+0000","last_change":"2024-09-05T02:00:12.783304+0000","last_active":"2024-09-05T02:02:27.200268+0000","last_peered":"2024-09-05T02:02:27.200268+0000","last_clean":"2024-09-05T02:02:27.200268+0000","last_became_active":"2024-09-05T02:00:12.782953+0000","last_became_peered":"2024-09-05T02:00:12.782953+0000","last_unstale":"2024-09-05T02:02:27.200268+0000","last_undegraded":"2024-09-05T02:02:27.200268+0000","last_fullsized":"2024-09-05T02:02:27.200268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:50:59.130705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362453+0000","last_change":"2024-09-05T02:00:21.418357+0000","last_active":"2024-09-05T02:02:27.362453+0000","last_peered":"2024-09-05T02:02:27.362453+0000","last_clean":"2024-09-05T02:02:27.362453+0000","last_became_active":"2024-09-05T02:00:21.417779+0000","last_became_peered":"2024-09-05T02:00:21.417779+0000","last_unstale":"2024-09-05T02:02:27.362453+0000","last_undegraded":"2024-09-05T02:02:27.362453+0000","last_fullsized":"2024-09-05T02:02:27.362453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:37:45.423447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200005+0000","last_change":"2024-09-05T02:00:14.362857+0000","last_active":"2024-09-05T02:02:27.200005+0000","last_peered":"2024-09-05T02:02:27.200005+0000","last_clean":"2024-09-05T02:02:27.200005+0000","last_became_active":"2024-09-05T02:00:14.362556+0000","last_became_peered":"2024-09-05T02:00:14.362556+0000","last_unstale":"2024-09-05T02:02:27.200005+0000","last_undegraded":"2024-09-05T02:02:27.200005+0000","last_fullsized":"2024-09-05T02:02:27.200005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:44:03.413055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362826+0000","last_change":"2024-09-05T02:00:12.512747+0000","last_active":"2024-09-05T02:02:27.362826+0000","last_peered":"2024-09-05T02:02:27.362826+0000","last_clean":"2024-09-05T02:02:27.362826+0000","last_became_active":"2024-09-05T02:00:12.512471+0000","last_became_peered":"2024-09-05T02:00:12.512471+0000","last_unstale":"2024-09-05T02:02:27.362826+0000","last_undegraded":"2024-09-05T02:02:27.362826+0000","last_fullsized":"2024-09-05T02:02:27.362826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:50.235020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204153+0000","last_change":"2024-09-05T02:00:21.413947+0000","last_active":"2024-09-05T02:02:27.204153+0000","last_peered":"2024-09-05T02:02:27.204153+0000","last_clean":"2024-09-05T02:02:27.204153+0000","last_became_active":"2024-09-05T02:00:21.413488+0000","last_became_peered":"2024-09-05T02:00:21.413488+0000","last_unstale":"2024-09-05T02:02:27.204153+0000","last_undegraded":"2024-09-05T02:02:27.204153+0000","last_fullsized":"2024-09-05T02:02:27.204153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:52:16.201325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207880+0000","last_change":"2024-09-05T02:00:14.367009+0000","last_active":"2024-09-05T02:02:27.207880+0000","last_peered":"2024-09-05T02:02:27.207880+0000","last_clean":"2024-09-05T02:02:27.207880+0000","last_became_active":"2024-09-05T02:00:14.366698+0000","last_became_peered":"2024-09-05T02:00:14.366698+0000","last_unstale":"2024-09-05T02:02:27.207880+0000","last_undegraded":"2024-09-05T02:02:27.207880+0000","last_fullsized":"2024-09-05T02:02:27.207880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:37:43.811300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'17","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359615+0000","last_change":"2024-09-05T02:00:12.346918+0000","last_active":"2024-09-05T02:02:27.359615+0000","last_peered":"2024-09-05T02:02:27.359615+0000","last_clean":"2024-09-05T02:02:27.359615+0000","last_became_active":"2024-09-05T02:00:12.346541+0000","last_became_peered":"2024-09-05T02:00:12.346541+0000","last_unstale":"2024-09-05T02:02:27.359615+0000","last_undegraded":"2024-09-05T02:02:27.359615+0000","last_fullsized":"2024-09-05T02:02:27.359615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:50.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207371+0000","last_change":"2024-09-05T02:00:21.413077+0000","last_active":"2024-09-05T02:02:27.207371+0000","last_peered":"2024-09-05T02:02:27.207371+0000","last_clean":"2024-09-05T02:02:27.207371+0000","last_became_active":"2024-09-05T02:00:21.412698+0000","last_became_peered":"2024-09-05T02:00:21.412698+0000","last_unstale":"2024-09-05T02:02:27.207371+0000","last_undegraded":"2024-09-05T02:02:27.207371+0000","last_fullsized":"2024-09-05T02:02:27.207371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:12:18.463614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359190+0000","last_change":"2024-09-05T02:00:14.363072+0000","last_active":"2024-09-05T02:02:27.359190+0000","last_peered":"2024-09-05T02:02:27.359190+0000","last_clean":"2024-09-05T02:02:27.359190+0000","last_became_active":"2024-09-05T02:00:14.361932+0000","last_became_peered":"2024-09-05T02:00:14.361932+0000","last_unstale":"2024-09-05T02:02:27.359190+0000","last_undegraded":"2024-09-05T02:02:27.359190+0000","last_fullsized":"2024-09-05T02:02:27.359190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:20:13.080963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208008+0000","last_change":"2024-09-05T02:00:12.783825+0000","last_active":"2024-09-05T02:02:27.208008+0000","last_peered":"2024-09-05T02:02:27.208008+0000","last_clean":"2024-09-05T02:02:27.208008+0000","last_became_active":"2024-09-05T02:00:12.783298+0000","last_became_peered":"2024-09-05T02:00:12.783298+0000","last_unstale":"2024-09-05T02:02:27.208008+0000","last_undegraded":"2024-09-05T02:02:27.208008+0000","last_fullsized":"2024-09-05T02:02:27.208008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:00:10.986318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205534+0000","last_change":"2024-09-05T02:00:21.416474+0000","last_active":"2024-09-05T02:02:27.205534+0000","last_peered":"2024-09-05T02:02:27.205534+0000","last_clean":"2024-09-05T02:02:27.205534+0000","last_became_active":"2024-09-05T02:00:21.416006+0000","last_became_peered":"2024-09-05T02:00:21.416006+0000","last_unstale":"2024-09-05T02:02:27.205534+0000","last_undegraded":"2024-09-05T02:02:27.205534+0000","last_fullsized":"2024-09-05T02:02:27.205534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:00:18.587895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361320+0000","last_change":"2024-09-05T02:00:14.362458+0000","last_active":"2024-09-05T02:02:27.361320+0000","last_peered":"2024-09-05T02:02:27.361320+0000","last_clean":"2024-09-05T02:02:27.361320+0000","last_became_active":"2024-09-05T02:00:14.362080+0000","last_became_peered":"2024-09-05T02:00:14.362080+0000","last_unstale":"2024-09-05T02:02:27.361320+0000","last_undegraded":"2024-09-05T02:02:27.361320+0000","last_fullsized":"2024-09-05T02:02:27.361320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:47:49.498468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359254+0000","last_change":"2024-09-05T02:00:12.783445+0000","last_active":"2024-09-05T02:02:27.359254+0000","last_peered":"2024-09-05T02:02:27.359254+0000","last_clean":"2024-09-05T02:02:27.359254+0000","last_became_active":"2024-09-05T02:00:12.783131+0000","last_became_peered":"2024-09-05T02:00:12.783131+0000","last_unstale":"2024-09-05T02:02:27.359254+0000","last_undegraded":"2024-09-05T02:02:27.359254+0000","last_fullsized":"2024-09-05T02:02:27.359254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:21:57.743629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208964+0000","last_change":"2024-09-05T02:00:21.418568+0000","last_active":"2024-09-05T02:02:27.208964+0000","last_peered":"2024-09-05T02:02:27.208964+0000","last_clean":"2024-09-05T02:02:27.208964+0000","last_became_active":"2024-09-05T02:00:21.418152+0000","last_became_peered":"2024-09-05T02:00:21.418152+0000","last_unstale":"2024-09-05T02:02:27.208964+0000","last_undegraded":"2024-09-05T02:02:27.208964+0000","last_fullsized":"2024-09-05T02:02:27.208964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:50:17.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361500+0000","last_change":"2024-09-05T02:00:14.364556+0000","last_active":"2024-09-05T02:02:27.361500+0000","last_peered":"2024-09-05T02:02:27.361500+0000","last_clean":"2024-09-05T02:02:27.361500+0000","last_became_active":"2024-09-05T02:00:14.364285+0000","last_became_peered":"2024-09-05T02:00:14.364285+0000","last_unstale":"2024-09-05T02:02:27.361500+0000","last_undegraded":"2024-09-05T02:02:27.361500+0000","last_fullsized":"2024-09-05T02:02:27.361500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:24:54.901510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:41.377697+0000","last_change":"2024-09-05T02:00:12.347993+0000","last_active":"2024-09-05T02:02:41.377697+0000","last_peered":"2024-09-05T02:02:41.377697+0000","last_clean":"2024-09-05T02:02:41.377697+0000","last_became_active":"2024-09-05T02:00:12.347656+0000","last_became_peered":"2024-09-05T02:00:12.347656+0000","last_unstale":"2024-09-05T02:02:41.377697+0000","last_undegraded":"2024-09-05T02:02:41.377697+0000","last_fullsized":"2024-09-05T02:02:41.377697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:31:23.820379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358999+0000","last_change":"2024-09-05T02:00:21.410786+0000","last_active":"2024-09-05T02:02:27.358999+0000","last_peered":"2024-09-05T02:02:27.358999+0000","last_clean":"2024-09-05T02:02:27.358999+0000","last_became_active":"2024-09-05T02:00:21.410440+0000","last_became_peered":"2024-09-05T02:00:21.410440+0000","last_unstale":"2024-09-05T02:02:27.358999+0000","last_undegraded":"2024-09-05T02:02:27.358999+0000","last_fullsized":"2024-09-05T02:02:27.358999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:22:14.644413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207789+0000","last_change":"2024-09-05T02:01:26.282147+0000","last_active":"2024-09-05T02:02:27.207789+0000","last_peered":"2024-09-05T02:02:27.207789+0000","last_clean":"2024-09-05T02:02:27.207789+0000","last_became_active":"2024-09-05T02:01:26.281792+0000","last_became_peered":"2024-09-05T02:01:26.281792+0000","last_unstale":"2024-09-05T02:02:27.207789+0000","last_undegraded":"2024-09-05T02:02:27.207789+0000","last_fullsized":"2024-09-05T02:02:27.207789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:10:43.403518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":144,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:36.372621+0000","last_change":"2024-09-05T02:00:12.783358+0000","last_active":"2024-09-05T02:02:36.372621+0000","last_peered":"2024-09-05T02:02:36.372621+0000","last_clean":"2024-09-05T02:02:36.372621+0000","last_became_active":"2024-09-05T02:00:12.782950+0000","last_became_peered":"2024-09-05T02:00:12.782950+0000","last_unstale":"2024-09-05T02:02:36.372621+0000","last_undegraded":"2024-09-05T02:02:36.372621+0000","last_fullsized":"2024-09-05T02:02:36.372621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:17:38.823019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359391+0000","last_change":"2024-09-05T02:00:21.416389+0000","last_active":"2024-09-05T02:02:27.359391+0000","last_peered":"2024-09-05T02:02:27.359391+0000","last_clean":"2024-09-05T02:02:27.359391+0000","last_became_active":"2024-09-05T02:00:21.416065+0000","last_became_peered":"2024-09-05T02:00:21.416065+0000","last_unstale":"2024-09-05T02:02:27.359391+0000","last_undegraded":"2024-09-05T02:02:27.359391+0000","last_fullsized":"2024-09-05T02:02:27.359391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:15.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358605+0000","last_change":"2024-09-05T02:00:14.361404+0000","last_active":"2024-09-05T02:02:27.358605+0000","last_peered":"2024-09-05T02:02:27.358605+0000","last_clean":"2024-09-05T02:02:27.358605+0000","last_became_active":"2024-09-05T02:00:14.361031+0000","last_became_peered":"2024-09-05T02:00:14.361031+0000","last_unstale":"2024-09-05T02:02:27.358605+0000","last_undegraded":"2024-09-05T02:02:27.358605+0000","last_fullsized":"2024-09-05T02:02:27.358605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:40.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.366300+0000","last_change":"2024-09-05T02:02:27.366300+0000","last_active":"2024-09-05T02:02:27.366300+0000","last_peered":"2024-09-05T02:02:27.366300+0000","last_clean":"2024-09-05T02:02:27.366300+0000","last_became_active":"2024-09-05T02:02:27.363278+0000","last_became_peered":"2024-09-05T02:02:27.363278+0000","last_unstale":"2024-09-05T02:02:27.366300+0000","last_undegraded":"2024-09-05T02:02:27.366300+0000","last_fullsized":"2024-09-05T02:02:27.366300+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:13.911137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207760+0000","last_change":"2024-09-05T02:01:26.510839+0000","last_active":"2024-09-05T02:02:27.207760+0000","last_peered":"2024-09-05T02:02:27.207760+0000","last_clean":"2024-09-05T02:02:27.207760+0000","last_became_active":"2024-09-05T02:01:26.509959+0000","last_became_peered":"2024-09-05T02:01:26.509959+0000","last_unstale":"2024-09-05T02:02:27.207760+0000","last_undegraded":"2024-09-05T02:02:27.207760+0000","last_fullsized":"2024-09-05T02:02:27.207760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:57.240190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207721+0000","last_change":"2024-09-05T02:00:14.359356+0000","last_active":"2024-09-05T02:02:27.207721+0000","last_peered":"2024-09-05T02:02:27.207721+0000","last_clean":"2024-09-05T02:02:27.207721+0000","last_became_active":"2024-09-05T02:00:14.359078+0000","last_became_peered":"2024-09-05T02:00:14.359078+0000","last_unstale":"2024-09-05T02:02:27.207721+0000","last_undegraded":"2024-09-05T02:02:27.207721+0000","last_fullsized":"2024-09-05T02:02:27.207721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:40:24.219962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362445+0000","last_change":"2024-09-05T02:00:12.349749+0000","last_active":"2024-09-05T02:02:27.362445+0000","last_peered":"2024-09-05T02:02:27.362445+0000","last_clean":"2024-09-05T02:02:27.362445+0000","last_became_active":"2024-09-05T02:00:12.349247+0000","last_became_peered":"2024-09-05T02:00:12.349247+0000","last_unstale":"2024-09-05T02:02:27.362445+0000","last_undegraded":"2024-09-05T02:02:27.362445+0000","last_fullsized":"2024-09-05T02:02:27.362445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:04:26.361209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203648+0000","last_change":"2024-09-05T01:58:57.552989+0000","last_active":"2024-09-05T02:02:27.203648+0000","last_peered":"2024-09-05T02:02:27.203648+0000","last_clean":"2024-09-05T02:02:27.203648+0000","last_became_active":"2024-09-05T01:58:57.552511+0000","last_became_peered":"2024-09-05T01:58:57.552511+0000","last_unstale":"2024-09-05T02:02:27.203648+0000","last_undegraded":"2024-09-05T02:02:27.203648+0000","last_fullsized":"2024-09-05T02:02:27.203648+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_clean_scrub_stamp":"2024-09-05T01:55:35.678445+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:53.711210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203681+0000","last_change":"2024-09-05T02:00:21.421456+0000","last_active":"2024-09-05T02:02:27.203681+0000","last_peered":"2024-09-05T02:02:27.203681+0000","last_clean":"2024-09-05T02:02:27.203681+0000","last_became_active":"2024-09-05T02:00:21.421074+0000","last_became_peered":"2024-09-05T02:00:21.421074+0000","last_unstale":"2024-09-05T02:02:27.203681+0000","last_undegraded":"2024-09-05T02:02:27.203681+0000","last_fullsized":"2024-09-05T02:02:27.203681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:46:00.660079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204029+0000","last_change":"2024-09-05T02:00:14.357638+0000","last_active":"2024-09-05T02:02:27.204029+0000","last_peered":"2024-09-05T02:02:27.204029+0000","last_clean":"2024-09-05T02:02:27.204029+0000","last_became_active":"2024-09-05T02:00:14.357125+0000","last_became_peered":"2024-09-05T02:00:14.357125+0000","last_unstale":"2024-09-05T02:02:27.204029+0000","last_undegraded":"2024-09-05T02:02:27.204029+0000","last_fullsized":"2024-09-05T02:02:27.204029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:02:04.343235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:38.376320+0000","last_change":"2024-09-05T02:00:12.783627+0000","last_active":"2024-09-05T02:02:38.376320+0000","last_peered":"2024-09-05T02:02:38.376320+0000","last_clean":"2024-09-05T02:02:38.376320+0000","last_became_active":"2024-09-05T02:00:12.783112+0000","last_became_peered":"2024-09-05T02:00:12.783112+0000","last_unstale":"2024-09-05T02:02:38.376320+0000","last_undegraded":"2024-09-05T02:02:38.376320+0000","last_fullsized":"2024-09-05T02:02:38.376320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:39:45.190311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200790+0000","last_change":"2024-09-05T02:00:21.413069+0000","last_active":"2024-09-05T02:02:27.200790+0000","last_peered":"2024-09-05T02:02:27.200790+0000","last_clean":"2024-09-05T02:02:27.200790+0000","last_became_active":"2024-09-05T02:00:21.412687+0000","last_became_peered":"2024-09-05T02:00:21.412687+0000","last_unstale":"2024-09-05T02:02:27.200790+0000","last_undegraded":"2024-09-05T02:02:27.200790+0000","last_fullsized":"2024-09-05T02:02:27.200790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:28.883071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203909+0000","last_change":"2024-09-05T02:00:14.363197+0000","last_active":"2024-09-05T02:02:27.203909+0000","last_peered":"2024-09-05T02:02:27.203909+0000","last_clean":"2024-09-05T02:02:27.203909+0000","last_became_active":"2024-09-05T02:00:14.362854+0000","last_became_peered":"2024-09-05T02:00:14.362854+0000","last_unstale":"2024-09-05T02:02:27.203909+0000","last_undegraded":"2024-09-05T02:02:27.203909+0000","last_fullsized":"2024-09-05T02:02:27.203909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:29:28.770604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208136+0000","last_change":"2024-09-05T02:00:12.348715+0000","last_active":"2024-09-05T02:02:27.208136+0000","last_peered":"2024-09-05T02:02:27.208136+0000","last_clean":"2024-09-05T02:02:27.208136+0000","last_became_active":"2024-09-05T02:00:12.348322+0000","last_became_peered":"2024-09-05T02:00:12.348322+0000","last_unstale":"2024-09-05T02:02:27.208136+0000","last_undegraded":"2024-09-05T02:02:27.208136+0000","last_fullsized":"2024-09-05T02:02:27.208136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:33:55.551080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199905+0000","last_change":"2024-09-05T02:01:26.509548+0000","last_active":"2024-09-05T02:02:27.199905+0000","last_peered":"2024-09-05T02:02:27.199905+0000","last_clean":"2024-09-05T02:02:27.199905+0000","last_became_active":"2024-09-05T02:01:26.508993+0000","last_became_peered":"2024-09-05T02:01:26.508993+0000","last_unstale":"2024-09-05T02:02:27.199905+0000","last_undegraded":"2024-09-05T02:02:27.199905+0000","last_fullsized":"2024-09-05T02:02:27.199905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:46.109685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203973+0000","last_change":"2024-09-05T02:00:14.355969+0000","last_active":"2024-09-05T02:02:27.203973+0000","last_peered":"2024-09-05T02:02:27.203973+0000","last_clean":"2024-09-05T02:02:27.203973+0000","last_became_active":"2024-09-05T02:00:14.355581+0000","last_became_peered":"2024-09-05T02:00:14.355581+0000","last_unstale":"2024-09-05T02:02:27.203973+0000","last_undegraded":"2024-09-05T02:02:27.203973+0000","last_fullsized":"2024-09-05T02:02:27.203973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:55:22.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359490+0000","last_change":"2024-09-05T02:00:12.348148+0000","last_active":"2024-09-05T02:02:27.359490+0000","last_peered":"2024-09-05T02:02:27.359490+0000","last_clean":"2024-09-05T02:02:27.359490+0000","last_became_active":"2024-09-05T02:00:12.347857+0000","last_became_peered":"2024-09-05T02:00:12.347857+0000","last_unstale":"2024-09-05T02:02:27.359490+0000","last_undegraded":"2024-09-05T02:02:27.359490+0000","last_fullsized":"2024-09-05T02:02:27.359490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:22:21.368082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207490+0000","last_change":"2024-09-05T02:00:21.417476+0000","last_active":"2024-09-05T02:02:27.207490+0000","last_peered":"2024-09-05T02:02:27.207490+0000","last_clean":"2024-09-05T02:02:27.207490+0000","last_became_active":"2024-09-05T02:00:21.417036+0000","last_became_peered":"2024-09-05T02:00:21.417036+0000","last_unstale":"2024-09-05T02:02:27.207490+0000","last_undegraded":"2024-09-05T02:02:27.207490+0000","last_fullsized":"2024-09-05T02:02:27.207490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:15:22.510948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205541+0000","last_change":"2024-09-05T02:00:14.358542+0000","last_active":"2024-09-05T02:02:27.205541+0000","last_peered":"2024-09-05T02:02:27.205541+0000","last_clean":"2024-09-05T02:02:27.205541+0000","last_became_active":"2024-09-05T02:00:14.358228+0000","last_became_peered":"2024-09-05T02:00:14.358228+0000","last_unstale":"2024-09-05T02:02:27.205541+0000","last_undegraded":"2024-09-05T02:02:27.205541+0000","last_fullsized":"2024-09-05T02:02:27.205541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:01.627453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208643+0000","last_change":"2024-09-05T02:00:12.345887+0000","last_active":"2024-09-05T02:02:27.208643+0000","last_peered":"2024-09-05T02:02:27.208643+0000","last_clean":"2024-09-05T02:02:27.208643+0000","last_became_active":"2024-09-05T02:00:12.345491+0000","last_became_peered":"2024-09-05T02:00:12.345491+0000","last_unstale":"2024-09-05T02:02:27.208643+0000","last_undegraded":"2024-09-05T02:02:27.208643+0000","last_fullsized":"2024-09-05T02:02:27.208643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:10:32.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207058+0000","last_change":"2024-09-05T02:00:21.418210+0000","last_active":"2024-09-05T02:02:27.207058+0000","last_peered":"2024-09-05T02:02:27.207058+0000","last_clean":"2024-09-05T02:02:27.207058+0000","last_became_active":"2024-09-05T02:00:21.417769+0000","last_became_peered":"2024-09-05T02:00:21.417769+0000","last_unstale":"2024-09-05T02:02:27.207058+0000","last_undegraded":"2024-09-05T02:02:27.207058+0000","last_fullsized":"2024-09-05T02:02:27.207058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:05.050499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207222+0000","last_change":"2024-09-05T02:00:14.367555+0000","last_active":"2024-09-05T02:02:27.207222+0000","last_peered":"2024-09-05T02:02:27.207222+0000","last_clean":"2024-09-05T02:02:27.207222+0000","last_became_active":"2024-09-05T02:00:14.367222+0000","last_became_peered":"2024-09-05T02:00:14.367222+0000","last_unstale":"2024-09-05T02:02:27.207222+0000","last_undegraded":"2024-09-05T02:02:27.207222+0000","last_fullsized":"2024-09-05T02:02:27.207222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:04:26.172141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207700+0000","last_change":"2024-09-05T02:01:26.282875+0000","last_active":"2024-09-05T02:02:27.207700+0000","last_peered":"2024-09-05T02:02:27.207700+0000","last_clean":"2024-09-05T02:02:27.207700+0000","last_became_active":"2024-09-05T02:01:26.271901+0000","last_became_peered":"2024-09-05T02:01:26.271901+0000","last_unstale":"2024-09-05T02:02:27.207700+0000","last_undegraded":"2024-09-05T02:02:27.207700+0000","last_fullsized":"2024-09-05T02:02:27.207700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:29:45.232870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359339+0000","last_change":"2024-09-05T02:00:21.412374+0000","last_active":"2024-09-05T02:02:27.359339+0000","last_peered":"2024-09-05T02:02:27.359339+0000","last_clean":"2024-09-05T02:02:27.359339+0000","last_became_active":"2024-09-05T02:00:21.411926+0000","last_became_peered":"2024-09-05T02:00:21.411926+0000","last_unstale":"2024-09-05T02:02:27.359339+0000","last_undegraded":"2024-09-05T02:02:27.359339+0000","last_fullsized":"2024-09-05T02:02:27.359339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:38:29.657795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203472+0000","last_change":"2024-09-05T02:01:26.269814+0000","last_active":"2024-09-05T02:02:27.203472+0000","last_peered":"2024-09-05T02:02:27.203472+0000","last_clean":"2024-09-05T02:02:27.203472+0000","last_became_active":"2024-09-05T02:01:26.269320+0000","last_became_peered":"2024-09-05T02:01:26.269320+0000","last_unstale":"2024-09-05T02:02:27.203472+0000","last_undegraded":"2024-09-05T02:02:27.203472+0000","last_fullsized":"2024-09-05T02:02:27.203472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:04:21.161986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:37.374155+0000","last_change":"2024-09-05T02:00:12.349175+0000","last_active":"2024-09-05T02:02:37.374155+0000","last_peered":"2024-09-05T02:02:37.374155+0000","last_clean":"2024-09-05T02:02:37.374155+0000","last_became_active":"2024-09-05T02:00:12.348647+0000","last_became_peered":"2024-09-05T02:00:12.348647+0000","last_unstale":"2024-09-05T02:02:37.374155+0000","last_undegraded":"2024-09-05T02:02:37.374155+0000","last_fullsized":"2024-09-05T02:02:37.374155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:38:13.363462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359005+0000","last_change":"2024-09-05T02:00:21.414252+0000","last_active":"2024-09-05T02:02:27.359005+0000","last_peered":"2024-09-05T02:02:27.359005+0000","last_clean":"2024-09-05T02:02:27.359005+0000","last_became_active":"2024-09-05T02:00:21.412483+0000","last_became_peered":"2024-09-05T02:00:21.412483+0000","last_unstale":"2024-09-05T02:02:27.359005+0000","last_undegraded":"2024-09-05T02:02:27.359005+0000","last_fullsized":"2024-09-05T02:02:27.359005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:58.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203867+0000","last_change":"2024-09-05T02:00:14.361018+0000","last_active":"2024-09-05T02:02:27.203867+0000","last_peered":"2024-09-05T02:02:27.203867+0000","last_clean":"2024-09-05T02:02:27.203867+0000","last_became_active":"2024-09-05T02:00:14.360591+0000","last_became_peered":"2024-09-05T02:00:14.360591+0000","last_unstale":"2024-09-05T02:02:27.203867+0000","last_undegraded":"2024-09-05T02:02:27.203867+0000","last_fullsized":"2024-09-05T02:02:27.203867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:49.175681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207571+0000","last_change":"2024-09-05T02:00:12.353257+0000","last_active":"2024-09-05T02:02:27.207571+0000","last_peered":"2024-09-05T02:02:27.207571+0000","last_clean":"2024-09-05T02:02:27.207571+0000","last_became_active":"2024-09-05T02:00:12.352906+0000","last_became_peered":"2024-09-05T02:00:12.352906+0000","last_unstale":"2024-09-05T02:02:27.207571+0000","last_undegraded":"2024-09-05T02:02:27.207571+0000","last_fullsized":"2024-09-05T02:02:27.207571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:53:52.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204956+0000","last_change":"2024-09-05T02:00:21.410346+0000","last_active":"2024-09-05T02:02:27.204956+0000","last_peered":"2024-09-05T02:02:27.204956+0000","last_clean":"2024-09-05T02:02:27.204956+0000","last_became_active":"2024-09-05T02:00:21.409946+0000","last_became_peered":"2024-09-05T02:00:21.409946+0000","last_unstale":"2024-09-05T02:02:27.204956+0000","last_undegraded":"2024-09-05T02:02:27.204956+0000","last_fullsized":"2024-09-05T02:02:27.204956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:56.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358918+0000","last_change":"2024-09-05T02:00:14.360265+0000","last_active":"2024-09-05T02:02:27.358918+0000","last_peered":"2024-09-05T02:02:27.358918+0000","last_clean":"2024-09-05T02:02:27.358918+0000","last_became_active":"2024-09-05T02:00:14.359958+0000","last_became_peered":"2024-09-05T02:00:14.359958+0000","last_unstale":"2024-09-05T02:02:27.358918+0000","last_undegraded":"2024-09-05T02:02:27.358918+0000","last_fullsized":"2024-09-05T02:02:27.358918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:15:45.211207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204648+0000","last_change":"2024-09-05T02:00:12.349443+0000","last_active":"2024-09-05T02:02:27.204648+0000","last_peered":"2024-09-05T02:02:27.204648+0000","last_clean":"2024-09-05T02:02:27.204648+0000","last_became_active":"2024-09-05T02:00:12.348988+0000","last_became_peered":"2024-09-05T02:00:12.348988+0000","last_unstale":"2024-09-05T02:02:27.204648+0000","last_undegraded":"2024-09-05T02:02:27.204648+0000","last_fullsized":"2024-09-05T02:02:27.204648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:14:21.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207693+0000","last_change":"2024-09-05T02:00:21.417662+0000","last_active":"2024-09-05T02:02:27.207693+0000","last_peered":"2024-09-05T02:02:27.207693+0000","last_clean":"2024-09-05T02:02:27.207693+0000","last_became_active":"2024-09-05T02:00:21.417238+0000","last_became_peered":"2024-09-05T02:00:21.417238+0000","last_unstale":"2024-09-05T02:02:27.207693+0000","last_undegraded":"2024-09-05T02:02:27.207693+0000","last_fullsized":"2024-09-05T02:02:27.207693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:05:59.304480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207729+0000","last_change":"2024-09-05T02:00:14.355928+0000","last_active":"2024-09-05T02:02:27.207729+0000","last_peered":"2024-09-05T02:02:27.207729+0000","last_clean":"2024-09-05T02:02:27.207729+0000","last_became_active":"2024-09-05T02:00:14.355576+0000","last_became_peered":"2024-09-05T02:00:14.355576+0000","last_unstale":"2024-09-05T02:02:27.207729+0000","last_undegraded":"2024-09-05T02:02:27.207729+0000","last_fullsized":"2024-09-05T02:02:27.207729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:22:50.095434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'37","reported_seq":286,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:38.375947+0000","last_change":"2024-09-05T02:00:12.352198+0000","last_active":"2024-09-05T02:02:38.375947+0000","last_peered":"2024-09-05T02:02:38.375947+0000","last_clean":"2024-09-05T02:02:38.375947+0000","last_became_active":"2024-09-05T02:00:12.351806+0000","last_became_peered":"2024-09-05T02:00:12.351806+0000","last_unstale":"2024-09-05T02:02:38.375947+0000","last_undegraded":"2024-09-05T02:02:38.375947+0000","last_fullsized":"2024-09-05T02:02:38.375947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.178160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62325,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":210,"num_read_kb":166,"num_write":39,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358666+0000","last_change":"2024-09-05T02:01:26.509575+0000","last_active":"2024-09-05T02:02:27.358666+0000","last_peered":"2024-09-05T02:02:27.358666+0000","last_clean":"2024-09-05T02:02:27.358666+0000","last_became_active":"2024-09-05T02:01:26.509108+0000","last_became_peered":"2024-09-05T02:01:26.509108+0000","last_unstale":"2024-09-05T02:02:27.358666+0000","last_undegraded":"2024-09-05T02:02:27.358666+0000","last_fullsized":"2024-09-05T02:02:27.358666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:55.503181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200586+0000","last_change":"2024-09-05T02:00:14.365372+0000","last_active":"2024-09-05T02:02:27.200586+0000","last_peered":"2024-09-05T02:02:27.200586+0000","last_clean":"2024-09-05T02:02:27.200586+0000","last_became_active":"2024-09-05T02:00:14.365064+0000","last_became_peered":"2024-09-05T02:00:14.365064+0000","last_unstale":"2024-09-05T02:02:27.200586+0000","last_undegraded":"2024-09-05T02:02:27.200586+0000","last_fullsized":"2024-09-05T02:02:27.200586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:02:29.378126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208012+0000","last_change":"2024-09-05T02:00:12.783487+0000","last_active":"2024-09-05T02:02:27.208012+0000","last_peered":"2024-09-05T02:02:27.208012+0000","last_clean":"2024-09-05T02:02:27.208012+0000","last_became_active":"2024-09-05T02:00:12.783087+0000","last_became_peered":"2024-09-05T02:00:12.783087+0000","last_unstale":"2024-09-05T02:02:27.208012+0000","last_undegraded":"2024-09-05T02:02:27.208012+0000","last_fullsized":"2024-09-05T02:02:27.208012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:08.574845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362092+0000","last_change":"2024-09-05T02:01:26.510122+0000","last_active":"2024-09-05T02:02:27.362092+0000","last_peered":"2024-09-05T02:02:27.362092+0000","last_clean":"2024-09-05T02:02:27.362092+0000","last_became_active":"2024-09-05T02:01:26.509427+0000","last_became_peered":"2024-09-05T02:01:26.509427+0000","last_unstale":"2024-09-05T02:02:27.362092+0000","last_undegraded":"2024-09-05T02:02:27.362092+0000","last_fullsized":"2024-09-05T02:02:27.362092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:50.139855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200186+0000","last_change":"2024-09-05T02:01:26.272703+0000","last_active":"2024-09-05T02:02:27.200186+0000","last_peered":"2024-09-05T02:02:27.200186+0000","last_clean":"2024-09-05T02:02:27.200186+0000","last_became_active":"2024-09-05T02:01:26.272440+0000","last_became_peered":"2024-09-05T02:01:26.272440+0000","last_unstale":"2024-09-05T02:02:27.200186+0000","last_undegraded":"2024-09-05T02:02:27.200186+0000","last_fullsized":"2024-09-05T02:02:27.200186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:29.855027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203783+0000","last_change":"2024-09-05T02:00:12.349362+0000","last_active":"2024-09-05T02:02:27.203783+0000","last_peered":"2024-09-05T02:02:27.203783+0000","last_clean":"2024-09-05T02:02:27.203783+0000","last_became_active":"2024-09-05T02:00:12.349016+0000","last_became_peered":"2024-09-05T02:00:12.349016+0000","last_unstale":"2024-09-05T02:02:27.203783+0000","last_undegraded":"2024-09-05T02:02:27.203783+0000","last_fullsized":"2024-09-05T02:02:27.203783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:37.603717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358193+0000","last_change":"2024-09-05T02:00:21.412942+0000","last_active":"2024-09-05T02:02:27.358193+0000","last_peered":"2024-09-05T02:02:27.358193+0000","last_clean":"2024-09-05T02:02:27.358193+0000","last_became_active":"2024-09-05T02:00:21.412057+0000","last_became_peered":"2024-09-05T02:00:21.412057+0000","last_unstale":"2024-09-05T02:02:27.358193+0000","last_undegraded":"2024-09-05T02:02:27.358193+0000","last_fullsized":"2024-09-05T02:02:27.358193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:59.872818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200666+0000","last_change":"2024-09-05T02:00:14.356636+0000","last_active":"2024-09-05T02:02:27.200666+0000","last_peered":"2024-09-05T02:02:27.200666+0000","last_clean":"2024-09-05T02:02:27.200666+0000","last_became_active":"2024-09-05T02:00:14.356321+0000","last_became_peered":"2024-09-05T02:00:14.356321+0000","last_unstale":"2024-09-05T02:02:27.200666+0000","last_undegraded":"2024-09-05T02:02:27.200666+0000","last_fullsized":"2024-09-05T02:02:27.200666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:02.112646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'6","reported_seq":245,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:38.376180+0000","last_change":"2024-09-05T02:00:12.353078+0000","last_active":"2024-09-05T02:02:38.376180+0000","last_peered":"2024-09-05T02:02:38.376180+0000","last_clean":"2024-09-05T02:02:38.376180+0000","last_became_active":"2024-09-05T02:00:12.352707+0000","last_became_peered":"2024-09-05T02:00:12.352707+0000","last_unstale":"2024-09-05T02:02:38.376180+0000","last_undegraded":"2024-09-05T02:02:38.376180+0000","last_fullsized":"2024-09-05T02:02:38.376180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:28:29.934304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207799+0000","last_change":"2024-09-05T02:00:21.421362+0000","last_active":"2024-09-05T02:02:27.207799+0000","last_peered":"2024-09-05T02:02:27.207799+0000","last_clean":"2024-09-05T02:02:27.207799+0000","last_became_active":"2024-09-05T02:00:21.420998+0000","last_became_peered":"2024-09-05T02:00:21.420998+0000","last_unstale":"2024-09-05T02:02:27.207799+0000","last_undegraded":"2024-09-05T02:02:27.207799+0000","last_fullsized":"2024-09-05T02:02:27.207799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:26:14.105489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199804+0000","last_change":"2024-09-05T02:00:21.416703+0000","last_active":"2024-09-05T02:02:27.199804+0000","last_peered":"2024-09-05T02:02:27.199804+0000","last_clean":"2024-09-05T02:02:27.199804+0000","last_became_active":"2024-09-05T02:00:21.415913+0000","last_became_peered":"2024-09-05T02:00:21.415913+0000","last_unstale":"2024-09-05T02:02:27.199804+0000","last_undegraded":"2024-09-05T02:02:27.199804+0000","last_fullsized":"2024-09-05T02:02:27.199804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:56.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358412+0000","last_change":"2024-09-05T02:00:14.355658+0000","last_active":"2024-09-05T02:02:27.358412+0000","last_peered":"2024-09-05T02:02:27.358412+0000","last_clean":"2024-09-05T02:02:27.358412+0000","last_became_active":"2024-09-05T02:00:14.355087+0000","last_became_peered":"2024-09-05T02:00:14.355087+0000","last_unstale":"2024-09-05T02:02:27.358412+0000","last_undegraded":"2024-09-05T02:02:27.358412+0000","last_fullsized":"2024-09-05T02:02:27.358412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:58:08.369027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207984+0000","last_change":"2024-09-05T02:00:12.348171+0000","last_active":"2024-09-05T02:02:27.207984+0000","last_peered":"2024-09-05T02:02:27.207984+0000","last_clean":"2024-09-05T02:02:27.207984+0000","last_became_active":"2024-09-05T02:00:12.347760+0000","last_became_peered":"2024-09-05T02:00:12.347760+0000","last_unstale":"2024-09-05T02:02:27.207984+0000","last_undegraded":"2024-09-05T02:02:27.207984+0000","last_fullsized":"2024-09-05T02:02:27.207984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:03:43.748458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205734+0000","last_change":"2024-09-05T02:00:21.413015+0000","last_active":"2024-09-05T02:02:27.205734+0000","last_peered":"2024-09-05T02:02:27.205734+0000","last_clean":"2024-09-05T02:02:27.205734+0000","last_became_active":"2024-09-05T02:00:21.412631+0000","last_became_peered":"2024-09-05T02:00:21.412631+0000","last_unstale":"2024-09-05T02:02:27.205734+0000","last_undegraded":"2024-09-05T02:02:27.205734+0000","last_fullsized":"2024-09-05T02:02:27.205734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:35.094273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207683+0000","last_change":"2024-09-05T02:00:14.354552+0000","last_active":"2024-09-05T02:02:27.207683+0000","last_peered":"2024-09-05T02:02:27.207683+0000","last_clean":"2024-09-05T02:02:27.207683+0000","last_became_active":"2024-09-05T02:00:14.354150+0000","last_became_peered":"2024-09-05T02:00:14.354150+0000","last_unstale":"2024-09-05T02:02:27.207683+0000","last_undegraded":"2024-09-05T02:02:27.207683+0000","last_fullsized":"2024-09-05T02:02:27.207683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:16:19.836148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358680+0000","last_change":"2024-09-05T02:00:12.342608+0000","last_active":"2024-09-05T02:02:27.358680+0000","last_peered":"2024-09-05T02:02:27.358680+0000","last_clean":"2024-09-05T02:02:27.358680+0000","last_became_active":"2024-09-05T02:00:12.342235+0000","last_became_peered":"2024-09-05T02:00:12.342235+0000","last_unstale":"2024-09-05T02:02:27.358680+0000","last_undegraded":"2024-09-05T02:02:27.358680+0000","last_fullsized":"2024-09-05T02:02:27.358680+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:54.568963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361875+0000","last_change":"2024-09-05T02:00:12.352247+0000","last_active":"2024-09-05T02:02:27.361875+0000","last_peered":"2024-09-05T02:02:27.361875+0000","last_clean":"2024-09-05T02:02:27.361875+0000","last_became_active":"2024-09-05T02:00:12.351897+0000","last_became_peered":"2024-09-05T02:00:12.351897+0000","last_unstale":"2024-09-05T02:02:27.361875+0000","last_undegraded":"2024-09-05T02:02:27.361875+0000","last_fullsized":"2024-09-05T02:02:27.361875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:48.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207250+0000","last_change":"2024-09-05T02:00:14.366415+0000","last_active":"2024-09-05T02:02:27.207250+0000","last_peered":"2024-09-05T02:02:27.207250+0000","last_clean":"2024-09-05T02:02:27.207250+0000","last_became_active":"2024-09-05T02:00:14.366103+0000","last_became_peered":"2024-09-05T02:00:14.366103+0000","last_unstale":"2024-09-05T02:02:27.207250+0000","last_undegraded":"2024-09-05T02:02:27.207250+0000","last_fullsized":"2024-09-05T02:02:27.207250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:22:21.585867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204377+0000","last_change":"2024-09-05T02:00:21.416879+0000","last_active":"2024-09-05T02:02:27.204377+0000","last_peered":"2024-09-05T02:02:27.204377+0000","last_clean":"2024-09-05T02:02:27.204377+0000","last_became_active":"2024-09-05T02:00:21.415887+0000","last_became_peered":"2024-09-05T02:00:21.415887+0000","last_unstale":"2024-09-05T02:02:27.204377+0000","last_undegraded":"2024-09-05T02:02:27.204377+0000","last_fullsized":"2024-09-05T02:02:27.204377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:58.106529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:38.376589+0000","last_change":"2024-09-05T02:00:12.354055+0000","last_active":"2024-09-05T02:02:38.376589+0000","last_peered":"2024-09-05T02:02:38.376589+0000","last_clean":"2024-09-05T02:02:38.376589+0000","last_became_active":"2024-09-05T02:00:12.353754+0000","last_became_peered":"2024-09-05T02:00:12.353754+0000","last_unstale":"2024-09-05T02:02:38.376589+0000","last_undegraded":"2024-09-05T02:02:38.376589+0000","last_fullsized":"2024-09-05T02:02:38.376589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:17.671079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.209049+0000","last_change":"2024-09-05T02:00:14.358646+0000","last_active":"2024-09-05T02:02:27.209049+0000","last_peered":"2024-09-05T02:02:27.209049+0000","last_clean":"2024-09-05T02:02:27.209049+0000","last_became_active":"2024-09-05T02:00:14.358337+0000","last_became_peered":"2024-09-05T02:00:14.358337+0000","last_unstale":"2024-09-05T02:02:27.209049+0000","last_undegraded":"2024-09-05T02:02:27.209049+0000","last_fullsized":"2024-09-05T02:02:27.209049+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:20:39.976731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204237+0000","last_change":"2024-09-05T02:00:21.416611+0000","last_active":"2024-09-05T02:02:27.204237+0000","last_peered":"2024-09-05T02:02:27.204237+0000","last_clean":"2024-09-05T02:02:27.204237+0000","last_became_active":"2024-09-05T02:00:21.416152+0000","last_became_peered":"2024-09-05T02:00:21.416152+0000","last_unstale":"2024-09-05T02:02:27.204237+0000","last_undegraded":"2024-09-05T02:02:27.204237+0000","last_fullsized":"2024-09-05T02:02:27.204237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:20.670383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205477+0000","last_change":"2024-09-05T02:00:14.359015+0000","last_active":"2024-09-05T02:02:27.205477+0000","last_peered":"2024-09-05T02:02:27.205477+0000","last_clean":"2024-09-05T02:02:27.205477+0000","last_became_active":"2024-09-05T02:00:14.358642+0000","last_became_peered":"2024-09-05T02:00:14.358642+0000","last_unstale":"2024-09-05T02:02:27.205477+0000","last_undegraded":"2024-09-05T02:02:27.205477+0000","last_fullsized":"2024-09-05T02:02:27.205477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:18:03.723033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204366+0000","last_change":"2024-09-05T02:00:14.362391+0000","last_active":"2024-09-05T02:02:27.204366+0000","last_peered":"2024-09-05T02:02:27.204366+0000","last_clean":"2024-09-05T02:02:27.204366+0000","last_became_active":"2024-09-05T02:00:14.362007+0000","last_became_peered":"2024-09-05T02:00:14.362007+0000","last_unstale":"2024-09-05T02:02:27.204366+0000","last_undegraded":"2024-09-05T02:02:27.204366+0000","last_fullsized":"2024-09-05T02:02:27.204366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:39:02.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204087+0000","last_change":"2024-09-05T02:01:26.269863+0000","last_active":"2024-09-05T02:02:27.204087+0000","last_peered":"2024-09-05T02:02:27.204087+0000","last_clean":"2024-09-05T02:02:27.204087+0000","last_became_active":"2024-09-05T02:01:26.269451+0000","last_became_peered":"2024-09-05T02:01:26.269451+0000","last_unstale":"2024-09-05T02:02:27.204087+0000","last_undegraded":"2024-09-05T02:02:27.204087+0000","last_fullsized":"2024-09-05T02:02:27.204087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:11.900726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199956+0000","last_change":"2024-09-05T02:00:14.357820+0000","last_active":"2024-09-05T02:02:27.199956+0000","last_peered":"2024-09-05T02:02:27.199956+0000","last_clean":"2024-09-05T02:02:27.199956+0000","last_became_active":"2024-09-05T02:00:14.357320+0000","last_became_peered":"2024-09-05T02:00:14.357320+0000","last_unstale":"2024-09-05T02:02:27.199956+0000","last_undegraded":"2024-09-05T02:02:27.199956+0000","last_fullsized":"2024-09-05T02:02:27.199956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:39.105192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359144+0000","last_change":"2024-09-05T02:00:14.356726+0000","last_active":"2024-09-05T02:02:27.359144+0000","last_peered":"2024-09-05T02:02:27.359144+0000","last_clean":"2024-09-05T02:02:27.359144+0000","last_became_active":"2024-09-05T02:00:14.356452+0000","last_became_peered":"2024-09-05T02:00:14.356452+0000","last_unstale":"2024-09-05T02:02:27.359144+0000","last_undegraded":"2024-09-05T02:02:27.359144+0000","last_fullsized":"2024-09-05T02:02:27.359144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:34:36.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205421+0000","last_change":"2024-09-05T02:00:14.357645+0000","last_active":"2024-09-05T02:02:27.205421+0000","last_peered":"2024-09-05T02:02:27.205421+0000","last_clean":"2024-09-05T02:02:27.205421+0000","last_became_active":"2024-09-05T02:00:14.357330+0000","last_became_peered":"2024-09-05T02:00:14.357330+0000","last_unstale":"2024-09-05T02:02:27.205421+0000","last_undegraded":"2024-09-05T02:02:27.205421+0000","last_fullsized":"2024-09-05T02:02:27.205421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:42.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204836+0000","last_change":"2024-09-05T02:00:14.363413+0000","last_active":"2024-09-05T02:02:27.204836+0000","last_peered":"2024-09-05T02:02:27.204836+0000","last_clean":"2024-09-05T02:02:27.204836+0000","last_became_active":"2024-09-05T02:00:14.362821+0000","last_became_peered":"2024-09-05T02:00:14.362821+0000","last_unstale":"2024-09-05T02:02:27.204836+0000","last_undegraded":"2024-09-05T02:02:27.204836+0000","last_fullsized":"2024-09-05T02:02:27.204836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:49:39.084171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207776+0000","last_change":"2024-09-05T02:00:14.366581+0000","last_active":"2024-09-05T02:02:27.207776+0000","last_peered":"2024-09-05T02:02:27.207776+0000","last_clean":"2024-09-05T02:02:27.207776+0000","last_became_active":"2024-09-05T02:00:14.366287+0000","last_became_peered":"2024-09-05T02:00:14.366287+0000","last_unstale":"2024-09-05T02:02:27.207776+0000","last_undegraded":"2024-09-05T02:02:27.207776+0000","last_fullsized":"2024-09-05T02:02:27.207776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:36:57.143489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207640+0000","last_change":"2024-09-05T02:00:14.360311+0000","last_active":"2024-09-05T02:02:27.207640+0000","last_peered":"2024-09-05T02:02:27.207640+0000","last_clean":"2024-09-05T02:02:27.207640+0000","last_became_active":"2024-09-05T02:00:14.360028+0000","last_became_peered":"2024-09-05T02:00:14.360028+0000","last_unstale":"2024-09-05T02:02:27.207640+0000","last_undegraded":"2024-09-05T02:02:27.207640+0000","last_fullsized":"2024-09-05T02:02:27.207640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:35:38.804555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207867+0000","last_change":"2024-09-05T02:00:14.365853+0000","last_active":"2024-09-05T02:02:27.207867+0000","last_peered":"2024-09-05T02:02:27.207867+0000","last_clean":"2024-09-05T02:02:27.207867+0000","last_became_active":"2024-09-05T02:00:14.365549+0000","last_became_peered":"2024-09-05T02:00:14.365549+0000","last_unstale":"2024-09-05T02:02:27.207867+0000","last_undegraded":"2024-09-05T02:02:27.207867+0000","last_fullsized":"2024-09-05T02:02:27.207867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:58:45.897592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200763+0000","last_change":"2024-09-05T02:00:14.367357+0000","last_active":"2024-09-05T02:02:27.200763+0000","last_peered":"2024-09-05T02:02:27.200763+0000","last_clean":"2024-09-05T02:02:27.200763+0000","last_became_active":"2024-09-05T02:00:14.366794+0000","last_became_peered":"2024-09-05T02:00:14.366794+0000","last_unstale":"2024-09-05T02:02:27.200763+0000","last_undegraded":"2024-09-05T02:02:27.200763+0000","last_fullsized":"2024-09-05T02:02:27.200763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:40:51.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359605+0000","last_change":"2024-09-05T02:00:14.356991+0000","last_active":"2024-09-05T02:02:27.359605+0000","last_peered":"2024-09-05T02:02:27.359605+0000","last_clean":"2024-09-05T02:02:27.359605+0000","last_became_active":"2024-09-05T02:00:14.356662+0000","last_became_peered":"2024-09-05T02:00:14.356662+0000","last_unstale":"2024-09-05T02:02:27.359605+0000","last_undegraded":"2024-09-05T02:02:27.359605+0000","last_fullsized":"2024-09-05T02:02:27.359605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:39:53.087448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205123+0000","last_change":"2024-09-05T02:00:14.357750+0000","last_active":"2024-09-05T02:02:27.205123+0000","last_peered":"2024-09-05T02:02:27.205123+0000","last_clean":"2024-09-05T02:02:27.205123+0000","last_became_active":"2024-09-05T02:00:14.357413+0000","last_became_peered":"2024-09-05T02:00:14.357413+0000","last_unstale":"2024-09-05T02:02:27.205123+0000","last_undegraded":"2024-09-05T02:02:27.205123+0000","last_fullsized":"2024-09-05T02:02:27.205123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:29:47.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204535+0000","last_change":"2024-09-05T02:00:14.364068+0000","last_active":"2024-09-05T02:02:27.204535+0000","last_peered":"2024-09-05T02:02:27.204535+0000","last_clean":"2024-09-05T02:02:27.204535+0000","last_became_active":"2024-09-05T02:00:14.363674+0000","last_became_peered":"2024-09-05T02:00:14.363674+0000","last_unstale":"2024-09-05T02:02:27.204535+0000","last_undegraded":"2024-09-05T02:02:27.204535+0000","last_fullsized":"2024-09-05T02:02:27.204535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:28:00.788668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359585+0000","last_change":"2024-09-05T02:00:14.355152+0000","last_active":"2024-09-05T02:02:27.359585+0000","last_peered":"2024-09-05T02:02:27.359585+0000","last_clean":"2024-09-05T02:02:27.359585+0000","last_became_active":"2024-09-05T02:00:14.354874+0000","last_became_peered":"2024-09-05T02:00:14.354874+0000","last_unstale":"2024-09-05T02:02:27.359585+0000","last_undegraded":"2024-09-05T02:02:27.359585+0000","last_fullsized":"2024-09-05T02:02:27.359585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:14:17.069049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.206078+0000","last_change":"2024-09-05T02:00:14.360241+0000","last_active":"2024-09-05T02:02:27.206078+0000","last_peered":"2024-09-05T02:02:27.206078+0000","last_clean":"2024-09-05T02:02:27.206078+0000","last_became_active":"2024-09-05T02:00:14.359648+0000","last_became_peered":"2024-09-05T02:00:14.359648+0000","last_unstale":"2024-09-05T02:02:27.206078+0000","last_undegraded":"2024-09-05T02:02:27.206078+0000","last_fullsized":"2024-09-05T02:02:27.206078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:15.060575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200191+0000","last_change":"2024-09-05T02:00:14.366094+0000","last_active":"2024-09-05T02:02:27.200191+0000","last_peered":"2024-09-05T02:02:27.200191+0000","last_clean":"2024-09-05T02:02:27.200191+0000","last_became_active":"2024-09-05T02:00:14.365788+0000","last_became_peered":"2024-09-05T02:00:14.365788+0000","last_unstale":"2024-09-05T02:02:27.200191+0000","last_undegraded":"2024-09-05T02:02:27.200191+0000","last_fullsized":"2024-09-05T02:02:27.200191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:23:00.909487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199671+0000","last_change":"2024-09-05T02:00:14.364651+0000","last_active":"2024-09-05T02:02:27.199671+0000","last_peered":"2024-09-05T02:02:27.199671+0000","last_clean":"2024-09-05T02:02:27.199671+0000","last_became_active":"2024-09-05T02:00:14.364365+0000","last_became_peered":"2024-09-05T02:00:14.364365+0000","last_unstale":"2024-09-05T02:02:27.199671+0000","last_undegraded":"2024-09-05T02:02:27.199671+0000","last_fullsized":"2024-09-05T02:02:27.199671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:14.978382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199677+0000","last_change":"2024-09-05T02:00:14.364761+0000","last_active":"2024-09-05T02:02:27.199677+0000","last_peered":"2024-09-05T02:02:27.199677+0000","last_clean":"2024-09-05T02:02:27.199677+0000","last_became_active":"2024-09-05T02:00:14.364204+0000","last_became_peered":"2024-09-05T02:00:14.364204+0000","last_unstale":"2024-09-05T02:02:27.199677+0000","last_undegraded":"2024-09-05T02:02:27.199677+0000","last_fullsized":"2024-09-05T02:02:27.199677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:16:51.076304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359513+0000","last_change":"2024-09-05T02:00:14.357632+0000","last_active":"2024-09-05T02:02:27.359513+0000","last_peered":"2024-09-05T02:02:27.359513+0000","last_clean":"2024-09-05T02:02:27.359513+0000","last_became_active":"2024-09-05T02:00:14.357348+0000","last_became_peered":"2024-09-05T02:00:14.357348+0000","last_unstale":"2024-09-05T02:02:27.359513+0000","last_undegraded":"2024-09-05T02:02:27.359513+0000","last_fullsized":"2024-09-05T02:02:27.359513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:11.163292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362670+0000","last_change":"2024-09-05T02:00:14.361484+0000","last_active":"2024-09-05T02:02:27.362670+0000","last_peered":"2024-09-05T02:02:27.362670+0000","last_clean":"2024-09-05T02:02:27.362670+0000","last_became_active":"2024-09-05T02:00:14.361156+0000","last_became_peered":"2024-09-05T02:00:14.361156+0000","last_unstale":"2024-09-05T02:02:27.362670+0000","last_undegraded":"2024-09-05T02:02:27.362670+0000","last_fullsized":"2024-09-05T02:02:27.362670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:40.124733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203916+0000","last_change":"2024-09-05T02:00:14.358325+0000","last_active":"2024-09-05T02:02:27.203916+0000","last_peered":"2024-09-05T02:02:27.203916+0000","last_clean":"2024-09-05T02:02:27.203916+0000","last_became_active":"2024-09-05T02:00:14.357993+0000","last_became_peered":"2024-09-05T02:00:14.357993+0000","last_unstale":"2024-09-05T02:02:27.203916+0000","last_undegraded":"2024-09-05T02:02:27.203916+0000","last_fullsized":"2024-09-05T02:02:27.203916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:18.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362551+0000","last_change":"2024-09-05T02:00:14.365291+0000","last_active":"2024-09-05T02:02:27.362551+0000","last_peered":"2024-09-05T02:02:27.362551+0000","last_clean":"2024-09-05T02:02:27.362551+0000","last_became_active":"2024-09-05T02:00:14.364937+0000","last_became_peered":"2024-09-05T02:00:14.364937+0000","last_unstale":"2024-09-05T02:02:27.362551+0000","last_undegraded":"2024-09-05T02:02:27.362551+0000","last_fullsized":"2024-09-05T02:02:27.362551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:42:57.712651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200788+0000","last_change":"2024-09-05T02:00:14.367603+0000","last_active":"2024-09-05T02:02:27.200788+0000","last_peered":"2024-09-05T02:02:27.200788+0000","last_clean":"2024-09-05T02:02:27.200788+0000","last_became_active":"2024-09-05T02:00:14.367267+0000","last_became_peered":"2024-09-05T02:00:14.367267+0000","last_unstale":"2024-09-05T02:02:27.200788+0000","last_undegraded":"2024-09-05T02:02:27.200788+0000","last_fullsized":"2024-09-05T02:02:27.200788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:51:34.595898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207492+0000","last_change":"2024-09-05T02:00:14.369499+0000","last_active":"2024-09-05T02:02:27.207492+0000","last_peered":"2024-09-05T02:02:27.207492+0000","last_clean":"2024-09-05T02:02:27.207492+0000","last_became_active":"2024-09-05T02:00:14.369169+0000","last_became_peered":"2024-09-05T02:00:14.369169+0000","last_unstale":"2024-09-05T02:02:27.207492+0000","last_undegraded":"2024-09-05T02:02:27.207492+0000","last_fullsized":"2024-09-05T02:02:27.207492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:17.161204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203441+0000","last_change":"2024-09-05T02:00:14.360320+0000","last_active":"2024-09-05T02:02:27.203441+0000","last_peered":"2024-09-05T02:02:27.203441+0000","last_clean":"2024-09-05T02:02:27.203441+0000","last_became_active":"2024-09-05T02:00:14.359991+0000","last_became_peered":"2024-09-05T02:00:14.359991+0000","last_unstale":"2024-09-05T02:02:27.203441+0000","last_undegraded":"2024-09-05T02:02:27.203441+0000","last_fullsized":"2024-09-05T02:02:27.203441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:03.721863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361323+0000","last_change":"2024-09-05T02:00:14.364660+0000","last_active":"2024-09-05T02:02:27.361323+0000","last_peered":"2024-09-05T02:02:27.361323+0000","last_clean":"2024-09-05T02:02:27.361323+0000","last_became_active":"2024-09-05T02:00:14.364313+0000","last_became_peered":"2024-09-05T02:00:14.364313+0000","last_unstale":"2024-09-05T02:02:27.361323+0000","last_undegraded":"2024-09-05T02:02:27.361323+0000","last_fullsized":"2024-09-05T02:02:27.361323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:47.496063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208875+0000","last_change":"2024-09-05T02:00:14.359853+0000","last_active":"2024-09-05T02:02:27.208875+0000","last_peered":"2024-09-05T02:02:27.208875+0000","last_clean":"2024-09-05T02:02:27.208875+0000","last_became_active":"2024-09-05T02:00:14.359543+0000","last_became_peered":"2024-09-05T02:00:14.359543+0000","last_unstale":"2024-09-05T02:02:27.208875+0000","last_undegraded":"2024-09-05T02:02:27.208875+0000","last_fullsized":"2024-09-05T02:02:27.208875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:30:29.415492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358876+0000","last_change":"2024-09-05T02:01:26.270888+0000","last_active":"2024-09-05T02:02:27.358876+0000","last_peered":"2024-09-05T02:02:27.358876+0000","last_clean":"2024-09-05T02:02:27.358876+0000","last_became_active":"2024-09-05T02:01:26.270607+0000","last_became_peered":"2024-09-05T02:01:26.270607+0000","last_unstale":"2024-09-05T02:02:27.358876+0000","last_undegraded":"2024-09-05T02:02:27.358876+0000","last_fullsized":"2024-09-05T02:02:27.358876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:16.563521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199910+0000","last_change":"2024-09-05T02:00:14.367191+0000","last_active":"2024-09-05T02:02:27.199910+0000","last_peered":"2024-09-05T02:02:27.199910+0000","last_clean":"2024-09-05T02:02:27.199910+0000","last_became_active":"2024-09-05T02:00:14.366831+0000","last_became_peered":"2024-09-05T02:00:14.366831+0000","last_unstale":"2024-09-05T02:02:27.199910+0000","last_undegraded":"2024-09-05T02:02:27.199910+0000","last_fullsized":"2024-09-05T02:02:27.199910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:55.690642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199853+0000","last_change":"2024-09-05T02:00:14.355774+0000","last_active":"2024-09-05T02:02:27.199853+0000","last_peered":"2024-09-05T02:02:27.199853+0000","last_clean":"2024-09-05T02:02:27.199853+0000","last_became_active":"2024-09-05T02:00:14.355293+0000","last_became_peered":"2024-09-05T02:00:14.355293+0000","last_unstale":"2024-09-05T02:02:27.199853+0000","last_undegraded":"2024-09-05T02:02:27.199853+0000","last_fullsized":"2024-09-05T02:02:27.199853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:07:00.547820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205824+0000","last_change":"2024-09-05T02:00:14.359978+0000","last_active":"2024-09-05T02:02:27.205824+0000","last_peered":"2024-09-05T02:02:27.205824+0000","last_clean":"2024-09-05T02:02:27.205824+0000","last_became_active":"2024-09-05T02:00:14.359209+0000","last_became_peered":"2024-09-05T02:00:14.359209+0000","last_unstale":"2024-09-05T02:02:27.205824+0000","last_undegraded":"2024-09-05T02:02:27.205824+0000","last_fullsized":"2024-09-05T02:02:27.205824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:27.848906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70310,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1047,"num_read_kb":598,"num_write":259,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":212415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":516663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1736704,"data_stored":1165270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1069056,"data_stored":520155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201339,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":516437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430271,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1114562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593797,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":511563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822729,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36980,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708172,"statfs":{"total":95995035648,"available":95957168128,"internally_reserved":0,"allocated":1667072,"data_stored":1108659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986255,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":515749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1052672,"data_stored":511271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444120,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1122304,"data_stored":576094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":575908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":514048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T02:02:44.626 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T02:02:44.626 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T02:02:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:44 smithi167 ceph-mon[35122]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:44.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:44 smithi139 ceph-mon[34262]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-05T02:02:44.875 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:45.325 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:45 smithi103 ceph-mon[30902]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:02:45.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 -- 172.21.15.103:0/863127511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c101de0 msgr2=0x7f656c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 --2- 172.21.15.103:0/863127511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c101de0 0x7f656c1021c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6554009900 tx=0x7f655402a410 comp rx=0 tx=0).stop 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 -- 172.21.15.103:0/863127511 shutdown_connections 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 --2- 172.21.15.103:0/863127511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656c106f10 0x7f656c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 --2- 172.21.15.103:0/863127511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f656c102790 0x7f656c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 --2- 172.21.15.103:0/863127511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c101de0 0x7f656c1021c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 -- 172.21.15.103:0/863127511 >> 172.21.15.103:0/863127511 conn(0x7f656c0fd8c0 msgr2=0x7f656c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 -- 172.21.15.103:0/863127511 shutdown_connections 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 -- 172.21.15.103:0/863127511 wait complete. 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.404+0000 7f6572230640 1 Processor -- start 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f6572230640 1 -- start start 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f6572230640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f656c101de0 0x7f656c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:45.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f6572230640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c102790 0x7f656c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f6572230640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656c106f10 0x7f656c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f6572230640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f656c111e20 con 0x7f656c102790 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f6572230640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f656c111ca0 con 0x7f656c101de0 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f6572230640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f656c111fa0 con 0x7f656c106f10 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c102790 0x7f656c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c102790 0x7f656c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55620/0 (socket says 172.21.15.103:55620) 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656affd640 1 -- 172.21.15.103:0/3721360554 learned_addr learned my addr 172.21.15.103:0/3721360554 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656affd640 1 -- 172.21.15.103:0/3721360554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656c106f10 msgr2=0x7f656c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656affd640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656c106f10 0x7f656c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656affd640 1 -- 172.21.15.103:0/3721360554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f656c101de0 msgr2=0x7f656c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656b7fe640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f656c101de0 0x7f656c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656affd640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f656c101de0 0x7f656c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656affd640 1 -- 172.21.15.103:0/3721360554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c1a67a0 con 0x7f656c102790 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656b7fe640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f656c101de0 0x7f656c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f656affd640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c102790 0x7f656c074a30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c920 tx=0x7f655c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f6568ff9640 1 -- 172.21.15.103:0/3721360554 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c007c70 con 0x7f656c102790 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.405+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656c1a6a90 con 0x7f656c102790 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.406+0000 7f6568ff9640 1 -- 172.21.15.103:0/3721360554 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f655c007e10 con 0x7f656c102790 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.406+0000 7f6568ff9640 1 -- 172.21.15.103:0/3721360554 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c018820 con 0x7f656c102790 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.406+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656c1a7020 con 0x7f656c102790 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.407+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f656c1021c0 con 0x7f656c102790 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.408+0000 7f6568ff9640 1 -- 172.21.15.103:0/3721360554 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f655c0189c0 con 0x7f656c102790 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.409+0000 7f6568ff9640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65580a2b80 0x7f65580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.409+0000 7f656b7fe640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65580a2b80 0x7f65580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.409+0000 7f6568ff9640 1 -- 172.21.15.103:0/3721360554 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f655c0cf890 con 0x7f656c102790 2024-09-05T02:02:45.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.409+0000 7f656b7fe640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65580a2b80 0x7f65580a5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6554000c00 tx=0x7f6554031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:45.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.412+0000 7f6568ff9640 1 -- 172.21.15.103:0/3721360554 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f655c014030 con 0x7f656c102790 2024-09-05T02:02:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.577+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 --> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f656c062630 con 0x7f65580a2b80 2024-09-05T02:02:45.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.585+0000 7f6568ff9640 1 -- 172.21.15.103:0/3721360554 <== mgr.14150 v2:172.21.15.103:6800/4267002703 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340542 (secure 0 0 0) 0x7f656c062630 con 0x7f65580a2b80 2024-09-05T02:02:45.590 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.588+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65580a2b80 msgr2=0x7f65580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.588+0000 7f6572230640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65580a2b80 0x7f65580a5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6554000c00 tx=0x7f6554031400 comp rx=0 tx=0).stop 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.588+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c102790 msgr2=0x7f656c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.589+0000 7f6572230640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c102790 0x7f656c074a30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c920 tx=0x7f655c00cde0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.589+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 shutdown_connections 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.589+0000 7f6572230640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65580a2b80 0x7f65580a5040 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.589+0000 7f6572230640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656c106f10 0x7f656c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.589+0000 7f6572230640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f656c102790 0x7f656c074a30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.589+0000 7f6572230640 1 --2- 172.21.15.103:0/3721360554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f656c101de0 0x7f656c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.589+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 >> 172.21.15.103:0/3721360554 conn(0x7f656c0fd8c0 msgr2=0x7f656c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.589+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 shutdown_connections 2024-09-05T02:02:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:45.589+0000 7f6572230640 1 -- 172.21.15.103:0/3721360554 wait complete. 2024-09-05T02:02:45.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:45 smithi167 ceph-mon[35122]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:02:45.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:45 smithi139 ceph-mon[34262]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:02:46.351 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-05T02:02:45.121418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666711,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1216,"num_read_kb":735,"num_write":455,"num_write_kb":3104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":873,"ondisk_log_size":873,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392736,"kb_used_data":14344,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549088,"statfs":{"total":1151940427776,"available":1151538266112,"internally_reserved":0,"allocated":14688256,"data_stored":8158472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6294,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":79,"num_write":14,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003304"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":159,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:42.379158+0000","last_change":"2024-09-05T02:00:12.347638+0000","last_active":"2024-09-05T02:02:42.379158+0000","last_peered":"2024-09-05T02:02:42.379158+0000","last_clean":"2024-09-05T02:02:42.379158+0000","last_became_active":"2024-09-05T02:00:12.347314+0000","last_became_peered":"2024-09-05T02:00:12.347314+0000","last_unstale":"2024-09-05T02:02:42.379158+0000","last_undegraded":"2024-09-05T02:02:42.379158+0000","last_fullsized":"2024-09-05T02:02:42.379158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:08:39.084117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207767+0000","last_change":"2024-09-05T02:00:14.360262+0000","last_active":"2024-09-05T02:02:27.207767+0000","last_peered":"2024-09-05T02:02:27.207767+0000","last_clean":"2024-09-05T02:02:27.207767+0000","last_became_active":"2024-09-05T02:00:14.359965+0000","last_became_peered":"2024-09-05T02:00:14.359965+0000","last_unstale":"2024-09-05T02:02:27.207767+0000","last_undegraded":"2024-09-05T02:02:27.207767+0000","last_fullsized":"2024-09-05T02:02:27.207767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:01:56.109169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203978+0000","last_change":"2024-09-05T02:00:21.410692+0000","last_active":"2024-09-05T02:02:27.203978+0000","last_peered":"2024-09-05T02:02:27.203978+0000","last_clean":"2024-09-05T02:02:27.203978+0000","last_became_active":"2024-09-05T02:00:21.410298+0000","last_became_peered":"2024-09-05T02:00:21.410298+0000","last_unstale":"2024-09-05T02:02:27.203978+0000","last_undegraded":"2024-09-05T02:02:27.203978+0000","last_fullsized":"2024-09-05T02:02:27.203978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:25:31.241700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207977+0000","last_change":"2024-09-05T02:00:12.783109+0000","last_active":"2024-09-05T02:02:27.207977+0000","last_peered":"2024-09-05T02:02:27.207977+0000","last_clean":"2024-09-05T02:02:27.207977+0000","last_became_active":"2024-09-05T02:00:12.782773+0000","last_became_peered":"2024-09-05T02:00:12.782773+0000","last_unstale":"2024-09-05T02:02:27.207977+0000","last_undegraded":"2024-09-05T02:02:27.207977+0000","last_fullsized":"2024-09-05T02:02:27.207977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:29:12.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200132+0000","last_change":"2024-09-05T02:00:14.370425+0000","last_active":"2024-09-05T02:02:27.200132+0000","last_peered":"2024-09-05T02:02:27.200132+0000","last_clean":"2024-09-05T02:02:27.200132+0000","last_became_active":"2024-09-05T02:00:14.370102+0000","last_became_peered":"2024-09-05T02:00:14.370102+0000","last_unstale":"2024-09-05T02:02:27.200132+0000","last_undegraded":"2024-09-05T02:02:27.200132+0000","last_fullsized":"2024-09-05T02:02:27.200132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:22:43.837418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362081+0000","last_change":"2024-09-05T02:00:21.413082+0000","last_active":"2024-09-05T02:02:27.362081+0000","last_peered":"2024-09-05T02:02:27.362081+0000","last_clean":"2024-09-05T02:02:27.362081+0000","last_became_active":"2024-09-05T02:00:21.412661+0000","last_became_peered":"2024-09-05T02:00:21.412661+0000","last_unstale":"2024-09-05T02:02:27.362081+0000","last_undegraded":"2024-09-05T02:02:27.362081+0000","last_fullsized":"2024-09-05T02:02:27.362081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:51:41.225573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207952+0000","last_change":"2024-09-05T02:00:12.353894+0000","last_active":"2024-09-05T02:02:27.207952+0000","last_peered":"2024-09-05T02:02:27.207952+0000","last_clean":"2024-09-05T02:02:27.207952+0000","last_became_active":"2024-09-05T02:00:12.353483+0000","last_became_peered":"2024-09-05T02:00:12.353483+0000","last_unstale":"2024-09-05T02:02:27.207952+0000","last_undegraded":"2024-09-05T02:02:27.207952+0000","last_fullsized":"2024-09-05T02:02:27.207952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:42.816578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200374+0000","last_change":"2024-09-05T02:00:14.356441+0000","last_active":"2024-09-05T02:02:27.200374+0000","last_peered":"2024-09-05T02:02:27.200374+0000","last_clean":"2024-09-05T02:02:27.200374+0000","last_became_active":"2024-09-05T02:00:14.356092+0000","last_became_peered":"2024-09-05T02:00:14.356092+0000","last_unstale":"2024-09-05T02:02:27.200374+0000","last_undegraded":"2024-09-05T02:02:27.200374+0000","last_fullsized":"2024-09-05T02:02:27.200374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:33:55.416968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362215+0000","last_change":"2024-09-05T02:00:21.418324+0000","last_active":"2024-09-05T02:02:27.362215+0000","last_peered":"2024-09-05T02:02:27.362215+0000","last_clean":"2024-09-05T02:02:27.362215+0000","last_became_active":"2024-09-05T02:00:21.417934+0000","last_became_peered":"2024-09-05T02:00:21.417934+0000","last_unstale":"2024-09-05T02:02:27.362215+0000","last_undegraded":"2024-09-05T02:02:27.362215+0000","last_fullsized":"2024-09-05T02:02:27.362215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:50.425823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208336+0000","last_change":"2024-09-05T02:00:12.348637+0000","last_active":"2024-09-05T02:02:27.208336+0000","last_peered":"2024-09-05T02:02:27.208336+0000","last_clean":"2024-09-05T02:02:27.208336+0000","last_became_active":"2024-09-05T02:00:12.348287+0000","last_became_peered":"2024-09-05T02:00:12.348287+0000","last_unstale":"2024-09-05T02:02:27.208336+0000","last_undegraded":"2024-09-05T02:02:27.208336+0000","last_fullsized":"2024-09-05T02:02:27.208336+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:52:58.368026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207802+0000","last_change":"2024-09-05T02:00:14.368414+0000","last_active":"2024-09-05T02:02:27.207802+0000","last_peered":"2024-09-05T02:02:27.207802+0000","last_clean":"2024-09-05T02:02:27.207802+0000","last_became_active":"2024-09-05T02:00:14.368018+0000","last_became_peered":"2024-09-05T02:00:14.368018+0000","last_unstale":"2024-09-05T02:02:27.207802+0000","last_undegraded":"2024-09-05T02:02:27.207802+0000","last_fullsized":"2024-09-05T02:02:27.207802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:08.246420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361924+0000","last_change":"2024-09-05T02:00:21.416649+0000","last_active":"2024-09-05T02:02:27.361924+0000","last_peered":"2024-09-05T02:02:27.361924+0000","last_clean":"2024-09-05T02:02:27.361924+0000","last_became_active":"2024-09-05T02:00:21.416247+0000","last_became_peered":"2024-09-05T02:00:21.416247+0000","last_unstale":"2024-09-05T02:02:27.361924+0000","last_undegraded":"2024-09-05T02:02:27.361924+0000","last_fullsized":"2024-09-05T02:02:27.361924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:10:00.115655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203731+0000","last_change":"2024-09-05T02:00:14.357719+0000","last_active":"2024-09-05T02:02:27.203731+0000","last_peered":"2024-09-05T02:02:27.203731+0000","last_clean":"2024-09-05T02:02:27.203731+0000","last_became_active":"2024-09-05T02:00:14.357211+0000","last_became_peered":"2024-09-05T02:00:14.357211+0000","last_unstale":"2024-09-05T02:02:27.203731+0000","last_undegraded":"2024-09-05T02:02:27.203731+0000","last_fullsized":"2024-09-05T02:02:27.203731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:27:15.654555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.214695+0000","last_change":"2024-09-05T02:02:27.214205+0000","last_active":"2024-09-05T02:02:27.214695+0000","last_peered":"2024-09-05T02:02:27.214695+0000","last_clean":"2024-09-05T02:02:27.214695+0000","last_became_active":"2024-09-05T02:02:27.209177+0000","last_became_peered":"2024-09-05T02:02:27.209177+0000","last_unstale":"2024-09-05T02:02:27.214695+0000","last_undegraded":"2024-09-05T02:02:27.214695+0000","last_fullsized":"2024-09-05T02:02:27.214695+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:06:12.009310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362275+0000","last_change":"2024-09-05T02:00:21.415203+0000","last_active":"2024-09-05T02:02:27.362275+0000","last_peered":"2024-09-05T02:02:27.362275+0000","last_clean":"2024-09-05T02:02:27.362275+0000","last_became_active":"2024-09-05T02:00:21.414767+0000","last_became_peered":"2024-09-05T02:00:21.414767+0000","last_unstale":"2024-09-05T02:02:27.362275+0000","last_undegraded":"2024-09-05T02:02:27.362275+0000","last_fullsized":"2024-09-05T02:02:27.362275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:01:48.398448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207759+0000","last_change":"2024-09-05T02:00:14.369417+0000","last_active":"2024-09-05T02:02:27.207759+0000","last_peered":"2024-09-05T02:02:27.207759+0000","last_clean":"2024-09-05T02:02:27.207759+0000","last_became_active":"2024-09-05T02:00:14.369093+0000","last_became_peered":"2024-09-05T02:00:14.369093+0000","last_unstale":"2024-09-05T02:02:27.207759+0000","last_undegraded":"2024-09-05T02:02:27.207759+0000","last_fullsized":"2024-09-05T02:02:27.207759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:01:34.239300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.217832+0000","last_change":"2024-09-05T02:02:27.217395+0000","last_active":"2024-09-05T02:02:27.217832+0000","last_peered":"2024-09-05T02:02:27.217832+0000","last_clean":"2024-09-05T02:02:27.217832+0000","last_became_active":"2024-09-05T02:02:27.210156+0000","last_became_peered":"2024-09-05T02:02:27.210156+0000","last_unstale":"2024-09-05T02:02:27.217832+0000","last_undegraded":"2024-09-05T02:02:27.217832+0000","last_fullsized":"2024-09-05T02:02:27.217832+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:26:02.816593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362455+0000","last_change":"2024-09-05T02:00:21.419070+0000","last_active":"2024-09-05T02:02:27.362455+0000","last_peered":"2024-09-05T02:02:27.362455+0000","last_clean":"2024-09-05T02:02:27.362455+0000","last_became_active":"2024-09-05T02:00:21.418315+0000","last_became_peered":"2024-09-05T02:00:21.418315+0000","last_unstale":"2024-09-05T02:02:27.362455+0000","last_undegraded":"2024-09-05T02:02:27.362455+0000","last_fullsized":"2024-09-05T02:02:27.362455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:17.974047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205618+0000","last_change":"2024-09-05T02:00:14.362965+0000","last_active":"2024-09-05T02:02:27.205618+0000","last_peered":"2024-09-05T02:02:27.205618+0000","last_clean":"2024-09-05T02:02:27.205618+0000","last_became_active":"2024-09-05T02:00:14.361842+0000","last_became_peered":"2024-09-05T02:00:14.361842+0000","last_unstale":"2024-09-05T02:02:27.205618+0000","last_undegraded":"2024-09-05T02:02:27.205618+0000","last_fullsized":"2024-09-05T02:02:27.205618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:45:10.565392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207940+0000","last_change":"2024-09-05T02:00:12.349368+0000","last_active":"2024-09-05T02:02:27.207940+0000","last_peered":"2024-09-05T02:02:27.207940+0000","last_clean":"2024-09-05T02:02:27.207940+0000","last_became_active":"2024-09-05T02:00:12.348990+0000","last_became_peered":"2024-09-05T02:00:12.348990+0000","last_unstale":"2024-09-05T02:02:27.207940+0000","last_undegraded":"2024-09-05T02:02:27.207940+0000","last_fullsized":"2024-09-05T02:02:27.207940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:45:04.998316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358957+0000","last_change":"2024-09-05T02:00:21.415467+0000","last_active":"2024-09-05T02:02:27.358957+0000","last_peered":"2024-09-05T02:02:27.358957+0000","last_clean":"2024-09-05T02:02:27.358957+0000","last_became_active":"2024-09-05T02:00:21.415138+0000","last_became_peered":"2024-09-05T02:00:21.415138+0000","last_unstale":"2024-09-05T02:02:27.358957+0000","last_undegraded":"2024-09-05T02:02:27.358957+0000","last_fullsized":"2024-09-05T02:02:27.358957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:51.221383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207499+0000","last_change":"2024-09-05T02:00:14.357961+0000","last_active":"2024-09-05T02:02:27.207499+0000","last_peered":"2024-09-05T02:02:27.207499+0000","last_clean":"2024-09-05T02:02:27.207499+0000","last_became_active":"2024-09-05T02:00:14.357624+0000","last_became_peered":"2024-09-05T02:00:14.357624+0000","last_unstale":"2024-09-05T02:02:27.207499+0000","last_undegraded":"2024-09-05T02:02:27.207499+0000","last_fullsized":"2024-09-05T02:02:27.207499+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:01.742852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200268+0000","last_change":"2024-09-05T02:00:12.783304+0000","last_active":"2024-09-05T02:02:27.200268+0000","last_peered":"2024-09-05T02:02:27.200268+0000","last_clean":"2024-09-05T02:02:27.200268+0000","last_became_active":"2024-09-05T02:00:12.782953+0000","last_became_peered":"2024-09-05T02:00:12.782953+0000","last_unstale":"2024-09-05T02:02:27.200268+0000","last_undegraded":"2024-09-05T02:02:27.200268+0000","last_fullsized":"2024-09-05T02:02:27.200268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:50:59.130705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362453+0000","last_change":"2024-09-05T02:00:21.418357+0000","last_active":"2024-09-05T02:02:27.362453+0000","last_peered":"2024-09-05T02:02:27.362453+0000","last_clean":"2024-09-05T02:02:27.362453+0000","last_became_active":"2024-09-05T02:00:21.417779+0000","last_became_peered":"2024-09-05T02:00:21.417779+0000","last_unstale":"2024-09-05T02:02:27.362453+0000","last_undegraded":"2024-09-05T02:02:27.362453+0000","last_fullsized":"2024-09-05T02:02:27.362453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:37:45.423447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200005+0000","last_change":"2024-09-05T02:00:14.362857+0000","last_active":"2024-09-05T02:02:27.200005+0000","last_peered":"2024-09-05T02:02:27.200005+0000","last_clean":"2024-09-05T02:02:27.200005+0000","last_became_active":"2024-09-05T02:00:14.362556+0000","last_became_peered":"2024-09-05T02:00:14.362556+0000","last_unstale":"2024-09-05T02:02:27.200005+0000","last_undegraded":"2024-09-05T02:02:27.200005+0000","last_fullsized":"2024-09-05T02:02:27.200005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:44:03.413055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362826+0000","last_change":"2024-09-05T02:00:12.512747+0000","last_active":"2024-09-05T02:02:27.362826+0000","last_peered":"2024-09-05T02:02:27.362826+0000","last_clean":"2024-09-05T02:02:27.362826+0000","last_became_active":"2024-09-05T02:00:12.512471+0000","last_became_peered":"2024-09-05T02:00:12.512471+0000","last_unstale":"2024-09-05T02:02:27.362826+0000","last_undegraded":"2024-09-05T02:02:27.362826+0000","last_fullsized":"2024-09-05T02:02:27.362826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:50.235020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204153+0000","last_change":"2024-09-05T02:00:21.413947+0000","last_active":"2024-09-05T02:02:27.204153+0000","last_peered":"2024-09-05T02:02:27.204153+0000","last_clean":"2024-09-05T02:02:27.204153+0000","last_became_active":"2024-09-05T02:00:21.413488+0000","last_became_peered":"2024-09-05T02:00:21.413488+0000","last_unstale":"2024-09-05T02:02:27.204153+0000","last_undegraded":"2024-09-05T02:02:27.204153+0000","last_fullsized":"2024-09-05T02:02:27.204153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:52:16.201325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207880+0000","last_change":"2024-09-05T02:00:14.367009+0000","last_active":"2024-09-05T02:02:27.207880+0000","last_peered":"2024-09-05T02:02:27.207880+0000","last_clean":"2024-09-05T02:02:27.207880+0000","last_became_active":"2024-09-05T02:00:14.366698+0000","last_became_peered":"2024-09-05T02:00:14.366698+0000","last_unstale":"2024-09-05T02:02:27.207880+0000","last_undegraded":"2024-09-05T02:02:27.207880+0000","last_fullsized":"2024-09-05T02:02:27.207880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:37:43.811300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'18","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359661+0000","last_change":"2024-09-05T02:00:12.346918+0000","last_active":"2024-09-05T02:02:27.359661+0000","last_peered":"2024-09-05T02:02:27.359661+0000","last_clean":"2024-09-05T02:02:27.359661+0000","last_became_active":"2024-09-05T02:00:12.346541+0000","last_became_peered":"2024-09-05T02:00:12.346541+0000","last_unstale":"2024-09-05T02:02:27.359661+0000","last_undegraded":"2024-09-05T02:02:27.359661+0000","last_fullsized":"2024-09-05T02:02:27.359661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:50.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207371+0000","last_change":"2024-09-05T02:00:21.413077+0000","last_active":"2024-09-05T02:02:27.207371+0000","last_peered":"2024-09-05T02:02:27.207371+0000","last_clean":"2024-09-05T02:02:27.207371+0000","last_became_active":"2024-09-05T02:00:21.412698+0000","last_became_peered":"2024-09-05T02:00:21.412698+0000","last_unstale":"2024-09-05T02:02:27.207371+0000","last_undegraded":"2024-09-05T02:02:27.207371+0000","last_fullsized":"2024-09-05T02:02:27.207371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:12:18.463614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359190+0000","last_change":"2024-09-05T02:00:14.363072+0000","last_active":"2024-09-05T02:02:27.359190+0000","last_peered":"2024-09-05T02:02:27.359190+0000","last_clean":"2024-09-05T02:02:27.359190+0000","last_became_active":"2024-09-05T02:00:14.361932+0000","last_became_peered":"2024-09-05T02:00:14.361932+0000","last_unstale":"2024-09-05T02:02:27.359190+0000","last_undegraded":"2024-09-05T02:02:27.359190+0000","last_fullsized":"2024-09-05T02:02:27.359190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:20:13.080963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208008+0000","last_change":"2024-09-05T02:00:12.783825+0000","last_active":"2024-09-05T02:02:27.208008+0000","last_peered":"2024-09-05T02:02:27.208008+0000","last_clean":"2024-09-05T02:02:27.208008+0000","last_became_active":"2024-09-05T02:00:12.783298+0000","last_became_peered":"2024-09-05T02:00:12.783298+0000","last_unstale":"2024-09-05T02:02:27.208008+0000","last_undegraded":"2024-09-05T02:02:27.208008+0000","last_fullsized":"2024-09-05T02:02:27.208008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:00:10.986318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205534+0000","last_change":"2024-09-05T02:00:21.416474+0000","last_active":"2024-09-05T02:02:27.205534+0000","last_peered":"2024-09-05T02:02:27.205534+0000","last_clean":"2024-09-05T02:02:27.205534+0000","last_became_active":"2024-09-05T02:00:21.416006+0000","last_became_peered":"2024-09-05T02:00:21.416006+0000","last_unstale":"2024-09-05T02:02:27.205534+0000","last_undegraded":"2024-09-05T02:02:27.205534+0000","last_fullsized":"2024-09-05T02:02:27.205534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:00:18.587895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361320+0000","last_change":"2024-09-05T02:00:14.362458+0000","last_active":"2024-09-05T02:02:27.361320+0000","last_peered":"2024-09-05T02:02:27.361320+0000","last_clean":"2024-09-05T02:02:27.361320+0000","last_became_active":"2024-09-05T02:00:14.362080+0000","last_became_peered":"2024-09-05T02:00:14.362080+0000","last_unstale":"2024-09-05T02:02:27.361320+0000","last_undegraded":"2024-09-05T02:02:27.361320+0000","last_fullsized":"2024-09-05T02:02:27.361320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:47:49.498468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359254+0000","last_change":"2024-09-05T02:00:12.783445+0000","last_active":"2024-09-05T02:02:27.359254+0000","last_peered":"2024-09-05T02:02:27.359254+0000","last_clean":"2024-09-05T02:02:27.359254+0000","last_became_active":"2024-09-05T02:00:12.783131+0000","last_became_peered":"2024-09-05T02:00:12.783131+0000","last_unstale":"2024-09-05T02:02:27.359254+0000","last_undegraded":"2024-09-05T02:02:27.359254+0000","last_fullsized":"2024-09-05T02:02:27.359254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:21:57.743629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208964+0000","last_change":"2024-09-05T02:00:21.418568+0000","last_active":"2024-09-05T02:02:27.208964+0000","last_peered":"2024-09-05T02:02:27.208964+0000","last_clean":"2024-09-05T02:02:27.208964+0000","last_became_active":"2024-09-05T02:00:21.418152+0000","last_became_peered":"2024-09-05T02:00:21.418152+0000","last_unstale":"2024-09-05T02:02:27.208964+0000","last_undegraded":"2024-09-05T02:02:27.208964+0000","last_fullsized":"2024-09-05T02:02:27.208964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:50:17.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361500+0000","last_change":"2024-09-05T02:00:14.364556+0000","last_active":"2024-09-05T02:02:27.361500+0000","last_peered":"2024-09-05T02:02:27.361500+0000","last_clean":"2024-09-05T02:02:27.361500+0000","last_became_active":"2024-09-05T02:00:14.364285+0000","last_became_peered":"2024-09-05T02:00:14.364285+0000","last_unstale":"2024-09-05T02:02:27.361500+0000","last_undegraded":"2024-09-05T02:02:27.361500+0000","last_fullsized":"2024-09-05T02:02:27.361500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:24:54.901510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:41.377697+0000","last_change":"2024-09-05T02:00:12.347993+0000","last_active":"2024-09-05T02:02:41.377697+0000","last_peered":"2024-09-05T02:02:41.377697+0000","last_clean":"2024-09-05T02:02:41.377697+0000","last_became_active":"2024-09-05T02:00:12.347656+0000","last_became_peered":"2024-09-05T02:00:12.347656+0000","last_unstale":"2024-09-05T02:02:41.377697+0000","last_undegraded":"2024-09-05T02:02:41.377697+0000","last_fullsized":"2024-09-05T02:02:41.377697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:31:23.820379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358999+0000","last_change":"2024-09-05T02:00:21.410786+0000","last_active":"2024-09-05T02:02:27.358999+0000","last_peered":"2024-09-05T02:02:27.358999+0000","last_clean":"2024-09-05T02:02:27.358999+0000","last_became_active":"2024-09-05T02:00:21.410440+0000","last_became_peered":"2024-09-05T02:00:21.410440+0000","last_unstale":"2024-09-05T02:02:27.358999+0000","last_undegraded":"2024-09-05T02:02:27.358999+0000","last_fullsized":"2024-09-05T02:02:27.358999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:22:14.644413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207789+0000","last_change":"2024-09-05T02:01:26.282147+0000","last_active":"2024-09-05T02:02:27.207789+0000","last_peered":"2024-09-05T02:02:27.207789+0000","last_clean":"2024-09-05T02:02:27.207789+0000","last_became_active":"2024-09-05T02:01:26.281792+0000","last_became_peered":"2024-09-05T02:01:26.281792+0000","last_unstale":"2024-09-05T02:02:27.207789+0000","last_undegraded":"2024-09-05T02:02:27.207789+0000","last_fullsized":"2024-09-05T02:02:27.207789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:10:43.403518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:41.377730+0000","last_change":"2024-09-05T02:00:12.783358+0000","last_active":"2024-09-05T02:02:41.377730+0000","last_peered":"2024-09-05T02:02:41.377730+0000","last_clean":"2024-09-05T02:02:41.377730+0000","last_became_active":"2024-09-05T02:00:12.782950+0000","last_became_peered":"2024-09-05T02:00:12.782950+0000","last_unstale":"2024-09-05T02:02:41.377730+0000","last_undegraded":"2024-09-05T02:02:41.377730+0000","last_fullsized":"2024-09-05T02:02:41.377730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:17:38.823019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359391+0000","last_change":"2024-09-05T02:00:21.416389+0000","last_active":"2024-09-05T02:02:27.359391+0000","last_peered":"2024-09-05T02:02:27.359391+0000","last_clean":"2024-09-05T02:02:27.359391+0000","last_became_active":"2024-09-05T02:00:21.416065+0000","last_became_peered":"2024-09-05T02:00:21.416065+0000","last_unstale":"2024-09-05T02:02:27.359391+0000","last_undegraded":"2024-09-05T02:02:27.359391+0000","last_fullsized":"2024-09-05T02:02:27.359391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:15.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358605+0000","last_change":"2024-09-05T02:00:14.361404+0000","last_active":"2024-09-05T02:02:27.358605+0000","last_peered":"2024-09-05T02:02:27.358605+0000","last_clean":"2024-09-05T02:02:27.358605+0000","last_became_active":"2024-09-05T02:00:14.361031+0000","last_became_peered":"2024-09-05T02:00:14.361031+0000","last_unstale":"2024-09-05T02:02:27.358605+0000","last_undegraded":"2024-09-05T02:02:27.358605+0000","last_fullsized":"2024-09-05T02:02:27.358605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:40.777710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.366300+0000","last_change":"2024-09-05T02:02:27.366300+0000","last_active":"2024-09-05T02:02:27.366300+0000","last_peered":"2024-09-05T02:02:27.366300+0000","last_clean":"2024-09-05T02:02:27.366300+0000","last_became_active":"2024-09-05T02:02:27.363278+0000","last_became_peered":"2024-09-05T02:02:27.363278+0000","last_unstale":"2024-09-05T02:02:27.366300+0000","last_undegraded":"2024-09-05T02:02:27.366300+0000","last_fullsized":"2024-09-05T02:02:27.366300+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:13.911137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207760+0000","last_change":"2024-09-05T02:01:26.510839+0000","last_active":"2024-09-05T02:02:27.207760+0000","last_peered":"2024-09-05T02:02:27.207760+0000","last_clean":"2024-09-05T02:02:27.207760+0000","last_became_active":"2024-09-05T02:01:26.509959+0000","last_became_peered":"2024-09-05T02:01:26.509959+0000","last_unstale":"2024-09-05T02:02:27.207760+0000","last_undegraded":"2024-09-05T02:02:27.207760+0000","last_fullsized":"2024-09-05T02:02:27.207760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:57.240190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207721+0000","last_change":"2024-09-05T02:00:14.359356+0000","last_active":"2024-09-05T02:02:27.207721+0000","last_peered":"2024-09-05T02:02:27.207721+0000","last_clean":"2024-09-05T02:02:27.207721+0000","last_became_active":"2024-09-05T02:00:14.359078+0000","last_became_peered":"2024-09-05T02:00:14.359078+0000","last_unstale":"2024-09-05T02:02:27.207721+0000","last_undegraded":"2024-09-05T02:02:27.207721+0000","last_fullsized":"2024-09-05T02:02:27.207721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:40:24.219962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362445+0000","last_change":"2024-09-05T02:00:12.349749+0000","last_active":"2024-09-05T02:02:27.362445+0000","last_peered":"2024-09-05T02:02:27.362445+0000","last_clean":"2024-09-05T02:02:27.362445+0000","last_became_active":"2024-09-05T02:00:12.349247+0000","last_became_peered":"2024-09-05T02:00:12.349247+0000","last_unstale":"2024-09-05T02:02:27.362445+0000","last_undegraded":"2024-09-05T02:02:27.362445+0000","last_fullsized":"2024-09-05T02:02:27.362445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:04:26.361209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203648+0000","last_change":"2024-09-05T01:58:57.552989+0000","last_active":"2024-09-05T02:02:27.203648+0000","last_peered":"2024-09-05T02:02:27.203648+0000","last_clean":"2024-09-05T02:02:27.203648+0000","last_became_active":"2024-09-05T01:58:57.552511+0000","last_became_peered":"2024-09-05T01:58:57.552511+0000","last_unstale":"2024-09-05T02:02:27.203648+0000","last_undegraded":"2024-09-05T02:02:27.203648+0000","last_fullsized":"2024-09-05T02:02:27.203648+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:55:35.678445+0000","last_clean_scrub_stamp":"2024-09-05T01:55:35.678445+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:53.711210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203681+0000","last_change":"2024-09-05T02:00:21.421456+0000","last_active":"2024-09-05T02:02:27.203681+0000","last_peered":"2024-09-05T02:02:27.203681+0000","last_clean":"2024-09-05T02:02:27.203681+0000","last_became_active":"2024-09-05T02:00:21.421074+0000","last_became_peered":"2024-09-05T02:00:21.421074+0000","last_unstale":"2024-09-05T02:02:27.203681+0000","last_undegraded":"2024-09-05T02:02:27.203681+0000","last_fullsized":"2024-09-05T02:02:27.203681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:46:00.660079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204029+0000","last_change":"2024-09-05T02:00:14.357638+0000","last_active":"2024-09-05T02:02:27.204029+0000","last_peered":"2024-09-05T02:02:27.204029+0000","last_clean":"2024-09-05T02:02:27.204029+0000","last_became_active":"2024-09-05T02:00:14.357125+0000","last_became_peered":"2024-09-05T02:00:14.357125+0000","last_unstale":"2024-09-05T02:02:27.204029+0000","last_undegraded":"2024-09-05T02:02:27.204029+0000","last_fullsized":"2024-09-05T02:02:27.204029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:02:04.343235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":158,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:43.382695+0000","last_change":"2024-09-05T02:00:12.783627+0000","last_active":"2024-09-05T02:02:43.382695+0000","last_peered":"2024-09-05T02:02:43.382695+0000","last_clean":"2024-09-05T02:02:43.382695+0000","last_became_active":"2024-09-05T02:00:12.783112+0000","last_became_peered":"2024-09-05T02:00:12.783112+0000","last_unstale":"2024-09-05T02:02:43.382695+0000","last_undegraded":"2024-09-05T02:02:43.382695+0000","last_fullsized":"2024-09-05T02:02:43.382695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:39:45.190311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200790+0000","last_change":"2024-09-05T02:00:21.413069+0000","last_active":"2024-09-05T02:02:27.200790+0000","last_peered":"2024-09-05T02:02:27.200790+0000","last_clean":"2024-09-05T02:02:27.200790+0000","last_became_active":"2024-09-05T02:00:21.412687+0000","last_became_peered":"2024-09-05T02:00:21.412687+0000","last_unstale":"2024-09-05T02:02:27.200790+0000","last_undegraded":"2024-09-05T02:02:27.200790+0000","last_fullsized":"2024-09-05T02:02:27.200790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:28.883071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203909+0000","last_change":"2024-09-05T02:00:14.363197+0000","last_active":"2024-09-05T02:02:27.203909+0000","last_peered":"2024-09-05T02:02:27.203909+0000","last_clean":"2024-09-05T02:02:27.203909+0000","last_became_active":"2024-09-05T02:00:14.362854+0000","last_became_peered":"2024-09-05T02:00:14.362854+0000","last_unstale":"2024-09-05T02:02:27.203909+0000","last_undegraded":"2024-09-05T02:02:27.203909+0000","last_fullsized":"2024-09-05T02:02:27.203909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:29:28.770604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208136+0000","last_change":"2024-09-05T02:00:12.348715+0000","last_active":"2024-09-05T02:02:27.208136+0000","last_peered":"2024-09-05T02:02:27.208136+0000","last_clean":"2024-09-05T02:02:27.208136+0000","last_became_active":"2024-09-05T02:00:12.348322+0000","last_became_peered":"2024-09-05T02:00:12.348322+0000","last_unstale":"2024-09-05T02:02:27.208136+0000","last_undegraded":"2024-09-05T02:02:27.208136+0000","last_fullsized":"2024-09-05T02:02:27.208136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:33:55.551080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199905+0000","last_change":"2024-09-05T02:01:26.509548+0000","last_active":"2024-09-05T02:02:27.199905+0000","last_peered":"2024-09-05T02:02:27.199905+0000","last_clean":"2024-09-05T02:02:27.199905+0000","last_became_active":"2024-09-05T02:01:26.508993+0000","last_became_peered":"2024-09-05T02:01:26.508993+0000","last_unstale":"2024-09-05T02:02:27.199905+0000","last_undegraded":"2024-09-05T02:02:27.199905+0000","last_fullsized":"2024-09-05T02:02:27.199905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:46.109685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203973+0000","last_change":"2024-09-05T02:00:14.355969+0000","last_active":"2024-09-05T02:02:27.203973+0000","last_peered":"2024-09-05T02:02:27.203973+0000","last_clean":"2024-09-05T02:02:27.203973+0000","last_became_active":"2024-09-05T02:00:14.355581+0000","last_became_peered":"2024-09-05T02:00:14.355581+0000","last_unstale":"2024-09-05T02:02:27.203973+0000","last_undegraded":"2024-09-05T02:02:27.203973+0000","last_fullsized":"2024-09-05T02:02:27.203973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:55:22.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359490+0000","last_change":"2024-09-05T02:00:12.348148+0000","last_active":"2024-09-05T02:02:27.359490+0000","last_peered":"2024-09-05T02:02:27.359490+0000","last_clean":"2024-09-05T02:02:27.359490+0000","last_became_active":"2024-09-05T02:00:12.347857+0000","last_became_peered":"2024-09-05T02:00:12.347857+0000","last_unstale":"2024-09-05T02:02:27.359490+0000","last_undegraded":"2024-09-05T02:02:27.359490+0000","last_fullsized":"2024-09-05T02:02:27.359490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:22:21.368082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207490+0000","last_change":"2024-09-05T02:00:21.417476+0000","last_active":"2024-09-05T02:02:27.207490+0000","last_peered":"2024-09-05T02:02:27.207490+0000","last_clean":"2024-09-05T02:02:27.207490+0000","last_became_active":"2024-09-05T02:00:21.417036+0000","last_became_peered":"2024-09-05T02:00:21.417036+0000","last_unstale":"2024-09-05T02:02:27.207490+0000","last_undegraded":"2024-09-05T02:02:27.207490+0000","last_fullsized":"2024-09-05T02:02:27.207490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:15:22.510948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205541+0000","last_change":"2024-09-05T02:00:14.358542+0000","last_active":"2024-09-05T02:02:27.205541+0000","last_peered":"2024-09-05T02:02:27.205541+0000","last_clean":"2024-09-05T02:02:27.205541+0000","last_became_active":"2024-09-05T02:00:14.358228+0000","last_became_peered":"2024-09-05T02:00:14.358228+0000","last_unstale":"2024-09-05T02:02:27.205541+0000","last_undegraded":"2024-09-05T02:02:27.205541+0000","last_fullsized":"2024-09-05T02:02:27.205541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:01.627453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:28.148975+0000","last_change":"2024-09-05T02:00:12.345887+0000","last_active":"2024-09-05T02:02:28.148975+0000","last_peered":"2024-09-05T02:02:28.148975+0000","last_clean":"2024-09-05T02:02:28.148975+0000","last_became_active":"2024-09-05T02:00:12.345491+0000","last_became_peered":"2024-09-05T02:00:12.345491+0000","last_unstale":"2024-09-05T02:02:28.148975+0000","last_undegraded":"2024-09-05T02:02:28.148975+0000","last_fullsized":"2024-09-05T02:02:28.148975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:10:32.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207058+0000","last_change":"2024-09-05T02:00:21.418210+0000","last_active":"2024-09-05T02:02:27.207058+0000","last_peered":"2024-09-05T02:02:27.207058+0000","last_clean":"2024-09-05T02:02:27.207058+0000","last_became_active":"2024-09-05T02:00:21.417769+0000","last_became_peered":"2024-09-05T02:00:21.417769+0000","last_unstale":"2024-09-05T02:02:27.207058+0000","last_undegraded":"2024-09-05T02:02:27.207058+0000","last_fullsized":"2024-09-05T02:02:27.207058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:05.050499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207222+0000","last_change":"2024-09-05T02:00:14.367555+0000","last_active":"2024-09-05T02:02:27.207222+0000","last_peered":"2024-09-05T02:02:27.207222+0000","last_clean":"2024-09-05T02:02:27.207222+0000","last_became_active":"2024-09-05T02:00:14.367222+0000","last_became_peered":"2024-09-05T02:00:14.367222+0000","last_unstale":"2024-09-05T02:02:27.207222+0000","last_undegraded":"2024-09-05T02:02:27.207222+0000","last_fullsized":"2024-09-05T02:02:27.207222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:04:26.172141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207700+0000","last_change":"2024-09-05T02:01:26.282875+0000","last_active":"2024-09-05T02:02:27.207700+0000","last_peered":"2024-09-05T02:02:27.207700+0000","last_clean":"2024-09-05T02:02:27.207700+0000","last_became_active":"2024-09-05T02:01:26.271901+0000","last_became_peered":"2024-09-05T02:01:26.271901+0000","last_unstale":"2024-09-05T02:02:27.207700+0000","last_undegraded":"2024-09-05T02:02:27.207700+0000","last_fullsized":"2024-09-05T02:02:27.207700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:29:45.232870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359339+0000","last_change":"2024-09-05T02:00:21.412374+0000","last_active":"2024-09-05T02:02:27.359339+0000","last_peered":"2024-09-05T02:02:27.359339+0000","last_clean":"2024-09-05T02:02:27.359339+0000","last_became_active":"2024-09-05T02:00:21.411926+0000","last_became_peered":"2024-09-05T02:00:21.411926+0000","last_unstale":"2024-09-05T02:02:27.359339+0000","last_undegraded":"2024-09-05T02:02:27.359339+0000","last_fullsized":"2024-09-05T02:02:27.359339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:38:29.657795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203472+0000","last_change":"2024-09-05T02:01:26.269814+0000","last_active":"2024-09-05T02:02:27.203472+0000","last_peered":"2024-09-05T02:02:27.203472+0000","last_clean":"2024-09-05T02:02:27.203472+0000","last_became_active":"2024-09-05T02:01:26.269320+0000","last_became_peered":"2024-09-05T02:01:26.269320+0000","last_unstale":"2024-09-05T02:02:27.203472+0000","last_undegraded":"2024-09-05T02:02:27.203472+0000","last_fullsized":"2024-09-05T02:02:27.203472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:04:21.161986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:42.380597+0000","last_change":"2024-09-05T02:00:12.349175+0000","last_active":"2024-09-05T02:02:42.380597+0000","last_peered":"2024-09-05T02:02:42.380597+0000","last_clean":"2024-09-05T02:02:42.380597+0000","last_became_active":"2024-09-05T02:00:12.348647+0000","last_became_peered":"2024-09-05T02:00:12.348647+0000","last_unstale":"2024-09-05T02:02:42.380597+0000","last_undegraded":"2024-09-05T02:02:42.380597+0000","last_fullsized":"2024-09-05T02:02:42.380597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:38:13.363462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359005+0000","last_change":"2024-09-05T02:00:21.414252+0000","last_active":"2024-09-05T02:02:27.359005+0000","last_peered":"2024-09-05T02:02:27.359005+0000","last_clean":"2024-09-05T02:02:27.359005+0000","last_became_active":"2024-09-05T02:00:21.412483+0000","last_became_peered":"2024-09-05T02:00:21.412483+0000","last_unstale":"2024-09-05T02:02:27.359005+0000","last_undegraded":"2024-09-05T02:02:27.359005+0000","last_fullsized":"2024-09-05T02:02:27.359005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:58.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203867+0000","last_change":"2024-09-05T02:00:14.361018+0000","last_active":"2024-09-05T02:02:27.203867+0000","last_peered":"2024-09-05T02:02:27.203867+0000","last_clean":"2024-09-05T02:02:27.203867+0000","last_became_active":"2024-09-05T02:00:14.360591+0000","last_became_peered":"2024-09-05T02:00:14.360591+0000","last_unstale":"2024-09-05T02:02:27.203867+0000","last_undegraded":"2024-09-05T02:02:27.203867+0000","last_fullsized":"2024-09-05T02:02:27.203867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:49.175681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207571+0000","last_change":"2024-09-05T02:00:12.353257+0000","last_active":"2024-09-05T02:02:27.207571+0000","last_peered":"2024-09-05T02:02:27.207571+0000","last_clean":"2024-09-05T02:02:27.207571+0000","last_became_active":"2024-09-05T02:00:12.352906+0000","last_became_peered":"2024-09-05T02:00:12.352906+0000","last_unstale":"2024-09-05T02:02:27.207571+0000","last_undegraded":"2024-09-05T02:02:27.207571+0000","last_fullsized":"2024-09-05T02:02:27.207571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:53:52.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204956+0000","last_change":"2024-09-05T02:00:21.410346+0000","last_active":"2024-09-05T02:02:27.204956+0000","last_peered":"2024-09-05T02:02:27.204956+0000","last_clean":"2024-09-05T02:02:27.204956+0000","last_became_active":"2024-09-05T02:00:21.409946+0000","last_became_peered":"2024-09-05T02:00:21.409946+0000","last_unstale":"2024-09-05T02:02:27.204956+0000","last_undegraded":"2024-09-05T02:02:27.204956+0000","last_fullsized":"2024-09-05T02:02:27.204956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:56.463837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358918+0000","last_change":"2024-09-05T02:00:14.360265+0000","last_active":"2024-09-05T02:02:27.358918+0000","last_peered":"2024-09-05T02:02:27.358918+0000","last_clean":"2024-09-05T02:02:27.358918+0000","last_became_active":"2024-09-05T02:00:14.359958+0000","last_became_peered":"2024-09-05T02:00:14.359958+0000","last_unstale":"2024-09-05T02:02:27.358918+0000","last_undegraded":"2024-09-05T02:02:27.358918+0000","last_fullsized":"2024-09-05T02:02:27.358918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:15:45.211207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204648+0000","last_change":"2024-09-05T02:00:12.349443+0000","last_active":"2024-09-05T02:02:27.204648+0000","last_peered":"2024-09-05T02:02:27.204648+0000","last_clean":"2024-09-05T02:02:27.204648+0000","last_became_active":"2024-09-05T02:00:12.348988+0000","last_became_peered":"2024-09-05T02:00:12.348988+0000","last_unstale":"2024-09-05T02:02:27.204648+0000","last_undegraded":"2024-09-05T02:02:27.204648+0000","last_fullsized":"2024-09-05T02:02:27.204648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:14:21.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207693+0000","last_change":"2024-09-05T02:00:21.417662+0000","last_active":"2024-09-05T02:02:27.207693+0000","last_peered":"2024-09-05T02:02:27.207693+0000","last_clean":"2024-09-05T02:02:27.207693+0000","last_became_active":"2024-09-05T02:00:21.417238+0000","last_became_peered":"2024-09-05T02:00:21.417238+0000","last_unstale":"2024-09-05T02:02:27.207693+0000","last_undegraded":"2024-09-05T02:02:27.207693+0000","last_fullsized":"2024-09-05T02:02:27.207693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:05:59.304480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207778+0000","last_change":"2024-09-05T02:00:14.355928+0000","last_active":"2024-09-05T02:02:27.207778+0000","last_peered":"2024-09-05T02:02:27.207778+0000","last_clean":"2024-09-05T02:02:27.207778+0000","last_became_active":"2024-09-05T02:00:14.355576+0000","last_became_peered":"2024-09-05T02:00:14.355576+0000","last_unstale":"2024-09-05T02:02:27.207778+0000","last_undegraded":"2024-09-05T02:02:27.207778+0000","last_fullsized":"2024-09-05T02:02:27.207778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:22:50.095434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:43.382918+0000","last_change":"2024-09-05T02:00:12.352198+0000","last_active":"2024-09-05T02:02:43.382918+0000","last_peered":"2024-09-05T02:02:43.382918+0000","last_clean":"2024-09-05T02:02:43.382918+0000","last_became_active":"2024-09-05T02:00:12.351806+0000","last_became_peered":"2024-09-05T02:00:12.351806+0000","last_unstale":"2024-09-05T02:02:43.382918+0000","last_undegraded":"2024-09-05T02:02:43.382918+0000","last_fullsized":"2024-09-05T02:02:43.382918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.178160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67134,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":221,"num_read_kb":172,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358714+0000","last_change":"2024-09-05T02:01:26.509575+0000","last_active":"2024-09-05T02:02:27.358714+0000","last_peered":"2024-09-05T02:02:27.358714+0000","last_clean":"2024-09-05T02:02:27.358714+0000","last_became_active":"2024-09-05T02:01:26.509108+0000","last_became_peered":"2024-09-05T02:01:26.509108+0000","last_unstale":"2024-09-05T02:02:27.358714+0000","last_undegraded":"2024-09-05T02:02:27.358714+0000","last_fullsized":"2024-09-05T02:02:27.358714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:55.503181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200586+0000","last_change":"2024-09-05T02:00:14.365372+0000","last_active":"2024-09-05T02:02:27.200586+0000","last_peered":"2024-09-05T02:02:27.200586+0000","last_clean":"2024-09-05T02:02:27.200586+0000","last_became_active":"2024-09-05T02:00:14.365064+0000","last_became_peered":"2024-09-05T02:00:14.365064+0000","last_unstale":"2024-09-05T02:02:27.200586+0000","last_undegraded":"2024-09-05T02:02:27.200586+0000","last_fullsized":"2024-09-05T02:02:27.200586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:02:29.378126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208012+0000","last_change":"2024-09-05T02:00:12.783487+0000","last_active":"2024-09-05T02:02:27.208012+0000","last_peered":"2024-09-05T02:02:27.208012+0000","last_clean":"2024-09-05T02:02:27.208012+0000","last_became_active":"2024-09-05T02:00:12.783087+0000","last_became_peered":"2024-09-05T02:00:12.783087+0000","last_unstale":"2024-09-05T02:02:27.208012+0000","last_undegraded":"2024-09-05T02:02:27.208012+0000","last_fullsized":"2024-09-05T02:02:27.208012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:08.574845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362145+0000","last_change":"2024-09-05T02:01:26.510122+0000","last_active":"2024-09-05T02:02:27.362145+0000","last_peered":"2024-09-05T02:02:27.362145+0000","last_clean":"2024-09-05T02:02:27.362145+0000","last_became_active":"2024-09-05T02:01:26.509427+0000","last_became_peered":"2024-09-05T02:01:26.509427+0000","last_unstale":"2024-09-05T02:02:27.362145+0000","last_undegraded":"2024-09-05T02:02:27.362145+0000","last_fullsized":"2024-09-05T02:02:27.362145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:50.139855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200186+0000","last_change":"2024-09-05T02:01:26.272703+0000","last_active":"2024-09-05T02:02:27.200186+0000","last_peered":"2024-09-05T02:02:27.200186+0000","last_clean":"2024-09-05T02:02:27.200186+0000","last_became_active":"2024-09-05T02:01:26.272440+0000","last_became_peered":"2024-09-05T02:01:26.272440+0000","last_unstale":"2024-09-05T02:02:27.200186+0000","last_undegraded":"2024-09-05T02:02:27.200186+0000","last_fullsized":"2024-09-05T02:02:27.200186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:29.855027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203783+0000","last_change":"2024-09-05T02:00:12.349362+0000","last_active":"2024-09-05T02:02:27.203783+0000","last_peered":"2024-09-05T02:02:27.203783+0000","last_clean":"2024-09-05T02:02:27.203783+0000","last_became_active":"2024-09-05T02:00:12.349016+0000","last_became_peered":"2024-09-05T02:00:12.349016+0000","last_unstale":"2024-09-05T02:02:27.203783+0000","last_undegraded":"2024-09-05T02:02:27.203783+0000","last_fullsized":"2024-09-05T02:02:27.203783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:37.603717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358251+0000","last_change":"2024-09-05T02:00:21.412942+0000","last_active":"2024-09-05T02:02:27.358251+0000","last_peered":"2024-09-05T02:02:27.358251+0000","last_clean":"2024-09-05T02:02:27.358251+0000","last_became_active":"2024-09-05T02:00:21.412057+0000","last_became_peered":"2024-09-05T02:00:21.412057+0000","last_unstale":"2024-09-05T02:02:27.358251+0000","last_undegraded":"2024-09-05T02:02:27.358251+0000","last_fullsized":"2024-09-05T02:02:27.358251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:59.872818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200666+0000","last_change":"2024-09-05T02:00:14.356636+0000","last_active":"2024-09-05T02:02:27.200666+0000","last_peered":"2024-09-05T02:02:27.200666+0000","last_clean":"2024-09-05T02:02:27.200666+0000","last_became_active":"2024-09-05T02:00:14.356321+0000","last_became_peered":"2024-09-05T02:00:14.356321+0000","last_unstale":"2024-09-05T02:02:27.200666+0000","last_undegraded":"2024-09-05T02:02:27.200666+0000","last_fullsized":"2024-09-05T02:02:27.200666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:02.112646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'8","reported_seq":257,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:43.382226+0000","last_change":"2024-09-05T02:00:12.353078+0000","last_active":"2024-09-05T02:02:43.382226+0000","last_peered":"2024-09-05T02:02:43.382226+0000","last_clean":"2024-09-05T02:02:43.382226+0000","last_became_active":"2024-09-05T02:00:12.352707+0000","last_became_peered":"2024-09-05T02:00:12.352707+0000","last_unstale":"2024-09-05T02:02:43.382226+0000","last_undegraded":"2024-09-05T02:02:43.382226+0000","last_fullsized":"2024-09-05T02:02:43.382226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:28:29.934304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207799+0000","last_change":"2024-09-05T02:00:21.421362+0000","last_active":"2024-09-05T02:02:27.207799+0000","last_peered":"2024-09-05T02:02:27.207799+0000","last_clean":"2024-09-05T02:02:27.207799+0000","last_became_active":"2024-09-05T02:00:21.420998+0000","last_became_peered":"2024-09-05T02:00:21.420998+0000","last_unstale":"2024-09-05T02:02:27.207799+0000","last_undegraded":"2024-09-05T02:02:27.207799+0000","last_fullsized":"2024-09-05T02:02:27.207799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:26:14.105489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199804+0000","last_change":"2024-09-05T02:00:21.416703+0000","last_active":"2024-09-05T02:02:27.199804+0000","last_peered":"2024-09-05T02:02:27.199804+0000","last_clean":"2024-09-05T02:02:27.199804+0000","last_became_active":"2024-09-05T02:00:21.415913+0000","last_became_peered":"2024-09-05T02:00:21.415913+0000","last_unstale":"2024-09-05T02:02:27.199804+0000","last_undegraded":"2024-09-05T02:02:27.199804+0000","last_fullsized":"2024-09-05T02:02:27.199804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:56.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358412+0000","last_change":"2024-09-05T02:00:14.355658+0000","last_active":"2024-09-05T02:02:27.358412+0000","last_peered":"2024-09-05T02:02:27.358412+0000","last_clean":"2024-09-05T02:02:27.358412+0000","last_became_active":"2024-09-05T02:00:14.355087+0000","last_became_peered":"2024-09-05T02:00:14.355087+0000","last_unstale":"2024-09-05T02:02:27.358412+0000","last_undegraded":"2024-09-05T02:02:27.358412+0000","last_fullsized":"2024-09-05T02:02:27.358412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:58:08.369027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207984+0000","last_change":"2024-09-05T02:00:12.348171+0000","last_active":"2024-09-05T02:02:27.207984+0000","last_peered":"2024-09-05T02:02:27.207984+0000","last_clean":"2024-09-05T02:02:27.207984+0000","last_became_active":"2024-09-05T02:00:12.347760+0000","last_became_peered":"2024-09-05T02:00:12.347760+0000","last_unstale":"2024-09-05T02:02:27.207984+0000","last_undegraded":"2024-09-05T02:02:27.207984+0000","last_fullsized":"2024-09-05T02:02:27.207984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:03:43.748458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205734+0000","last_change":"2024-09-05T02:00:21.413015+0000","last_active":"2024-09-05T02:02:27.205734+0000","last_peered":"2024-09-05T02:02:27.205734+0000","last_clean":"2024-09-05T02:02:27.205734+0000","last_became_active":"2024-09-05T02:00:21.412631+0000","last_became_peered":"2024-09-05T02:00:21.412631+0000","last_unstale":"2024-09-05T02:02:27.205734+0000","last_undegraded":"2024-09-05T02:02:27.205734+0000","last_fullsized":"2024-09-05T02:02:27.205734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:35.094273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207683+0000","last_change":"2024-09-05T02:00:14.354552+0000","last_active":"2024-09-05T02:02:27.207683+0000","last_peered":"2024-09-05T02:02:27.207683+0000","last_clean":"2024-09-05T02:02:27.207683+0000","last_became_active":"2024-09-05T02:00:14.354150+0000","last_became_peered":"2024-09-05T02:00:14.354150+0000","last_unstale":"2024-09-05T02:02:27.207683+0000","last_undegraded":"2024-09-05T02:02:27.207683+0000","last_fullsized":"2024-09-05T02:02:27.207683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:16:19.836148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358680+0000","last_change":"2024-09-05T02:00:12.342608+0000","last_active":"2024-09-05T02:02:27.358680+0000","last_peered":"2024-09-05T02:02:27.358680+0000","last_clean":"2024-09-05T02:02:27.358680+0000","last_became_active":"2024-09-05T02:00:12.342235+0000","last_became_peered":"2024-09-05T02:00:12.342235+0000","last_unstale":"2024-09-05T02:02:27.358680+0000","last_undegraded":"2024-09-05T02:02:27.358680+0000","last_fullsized":"2024-09-05T02:02:27.358680+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:54.568963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361944+0000","last_change":"2024-09-05T02:00:12.352247+0000","last_active":"2024-09-05T02:02:27.361944+0000","last_peered":"2024-09-05T02:02:27.361944+0000","last_clean":"2024-09-05T02:02:27.361944+0000","last_became_active":"2024-09-05T02:00:12.351897+0000","last_became_peered":"2024-09-05T02:00:12.351897+0000","last_unstale":"2024-09-05T02:02:27.361944+0000","last_undegraded":"2024-09-05T02:02:27.361944+0000","last_fullsized":"2024-09-05T02:02:27.361944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:15:48.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207250+0000","last_change":"2024-09-05T02:00:14.366415+0000","last_active":"2024-09-05T02:02:27.207250+0000","last_peered":"2024-09-05T02:02:27.207250+0000","last_clean":"2024-09-05T02:02:27.207250+0000","last_became_active":"2024-09-05T02:00:14.366103+0000","last_became_peered":"2024-09-05T02:00:14.366103+0000","last_unstale":"2024-09-05T02:02:27.207250+0000","last_undegraded":"2024-09-05T02:02:27.207250+0000","last_fullsized":"2024-09-05T02:02:27.207250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:22:21.585867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204377+0000","last_change":"2024-09-05T02:00:21.416879+0000","last_active":"2024-09-05T02:02:27.204377+0000","last_peered":"2024-09-05T02:02:27.204377+0000","last_clean":"2024-09-05T02:02:27.204377+0000","last_became_active":"2024-09-05T02:00:21.415887+0000","last_became_peered":"2024-09-05T02:00:21.415887+0000","last_unstale":"2024-09-05T02:02:27.204377+0000","last_undegraded":"2024-09-05T02:02:27.204377+0000","last_fullsized":"2024-09-05T02:02:27.204377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:58.106529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:38.376589+0000","last_change":"2024-09-05T02:00:12.354055+0000","last_active":"2024-09-05T02:02:38.376589+0000","last_peered":"2024-09-05T02:02:38.376589+0000","last_clean":"2024-09-05T02:02:38.376589+0000","last_became_active":"2024-09-05T02:00:12.353754+0000","last_became_peered":"2024-09-05T02:00:12.353754+0000","last_unstale":"2024-09-05T02:02:38.376589+0000","last_undegraded":"2024-09-05T02:02:38.376589+0000","last_fullsized":"2024-09-05T02:02:38.376589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:11.333637+0000","last_clean_scrub_stamp":"2024-09-05T02:00:11.333637+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:17.671079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.209049+0000","last_change":"2024-09-05T02:00:14.358646+0000","last_active":"2024-09-05T02:02:27.209049+0000","last_peered":"2024-09-05T02:02:27.209049+0000","last_clean":"2024-09-05T02:02:27.209049+0000","last_became_active":"2024-09-05T02:00:14.358337+0000","last_became_peered":"2024-09-05T02:00:14.358337+0000","last_unstale":"2024-09-05T02:02:27.209049+0000","last_undegraded":"2024-09-05T02:02:27.209049+0000","last_fullsized":"2024-09-05T02:02:27.209049+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:20:39.976731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204237+0000","last_change":"2024-09-05T02:00:21.416611+0000","last_active":"2024-09-05T02:02:27.204237+0000","last_peered":"2024-09-05T02:02:27.204237+0000","last_clean":"2024-09-05T02:02:27.204237+0000","last_became_active":"2024-09-05T02:00:21.416152+0000","last_became_peered":"2024-09-05T02:00:21.416152+0000","last_unstale":"2024-09-05T02:02:27.204237+0000","last_undegraded":"2024-09-05T02:02:27.204237+0000","last_fullsized":"2024-09-05T02:02:27.204237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:20.386329+0000","last_clean_scrub_stamp":"2024-09-05T02:00:20.386329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:20.670383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205477+0000","last_change":"2024-09-05T02:00:14.359015+0000","last_active":"2024-09-05T02:02:27.205477+0000","last_peered":"2024-09-05T02:02:27.205477+0000","last_clean":"2024-09-05T02:02:27.205477+0000","last_became_active":"2024-09-05T02:00:14.358642+0000","last_became_peered":"2024-09-05T02:00:14.358642+0000","last_unstale":"2024-09-05T02:02:27.205477+0000","last_undegraded":"2024-09-05T02:02:27.205477+0000","last_fullsized":"2024-09-05T02:02:27.205477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:18:03.723033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204366+0000","last_change":"2024-09-05T02:00:14.362391+0000","last_active":"2024-09-05T02:02:27.204366+0000","last_peered":"2024-09-05T02:02:27.204366+0000","last_clean":"2024-09-05T02:02:27.204366+0000","last_became_active":"2024-09-05T02:00:14.362007+0000","last_became_peered":"2024-09-05T02:00:14.362007+0000","last_unstale":"2024-09-05T02:02:27.204366+0000","last_undegraded":"2024-09-05T02:02:27.204366+0000","last_fullsized":"2024-09-05T02:02:27.204366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:39:02.743880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204087+0000","last_change":"2024-09-05T02:01:26.269863+0000","last_active":"2024-09-05T02:02:27.204087+0000","last_peered":"2024-09-05T02:02:27.204087+0000","last_clean":"2024-09-05T02:02:27.204087+0000","last_became_active":"2024-09-05T02:01:26.269451+0000","last_became_peered":"2024-09-05T02:01:26.269451+0000","last_unstale":"2024-09-05T02:02:27.204087+0000","last_undegraded":"2024-09-05T02:02:27.204087+0000","last_fullsized":"2024-09-05T02:02:27.204087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:11.900726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199956+0000","last_change":"2024-09-05T02:00:14.357820+0000","last_active":"2024-09-05T02:02:27.199956+0000","last_peered":"2024-09-05T02:02:27.199956+0000","last_clean":"2024-09-05T02:02:27.199956+0000","last_became_active":"2024-09-05T02:00:14.357320+0000","last_became_peered":"2024-09-05T02:00:14.357320+0000","last_unstale":"2024-09-05T02:02:27.199956+0000","last_undegraded":"2024-09-05T02:02:27.199956+0000","last_fullsized":"2024-09-05T02:02:27.199956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:39.105192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359144+0000","last_change":"2024-09-05T02:00:14.356726+0000","last_active":"2024-09-05T02:02:27.359144+0000","last_peered":"2024-09-05T02:02:27.359144+0000","last_clean":"2024-09-05T02:02:27.359144+0000","last_became_active":"2024-09-05T02:00:14.356452+0000","last_became_peered":"2024-09-05T02:00:14.356452+0000","last_unstale":"2024-09-05T02:02:27.359144+0000","last_undegraded":"2024-09-05T02:02:27.359144+0000","last_fullsized":"2024-09-05T02:02:27.359144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:34:36.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205421+0000","last_change":"2024-09-05T02:00:14.357645+0000","last_active":"2024-09-05T02:02:27.205421+0000","last_peered":"2024-09-05T02:02:27.205421+0000","last_clean":"2024-09-05T02:02:27.205421+0000","last_became_active":"2024-09-05T02:00:14.357330+0000","last_became_peered":"2024-09-05T02:00:14.357330+0000","last_unstale":"2024-09-05T02:02:27.205421+0000","last_undegraded":"2024-09-05T02:02:27.205421+0000","last_fullsized":"2024-09-05T02:02:27.205421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:42.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204836+0000","last_change":"2024-09-05T02:00:14.363413+0000","last_active":"2024-09-05T02:02:27.204836+0000","last_peered":"2024-09-05T02:02:27.204836+0000","last_clean":"2024-09-05T02:02:27.204836+0000","last_became_active":"2024-09-05T02:00:14.362821+0000","last_became_peered":"2024-09-05T02:00:14.362821+0000","last_unstale":"2024-09-05T02:02:27.204836+0000","last_undegraded":"2024-09-05T02:02:27.204836+0000","last_fullsized":"2024-09-05T02:02:27.204836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:49:39.084171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207776+0000","last_change":"2024-09-05T02:00:14.366581+0000","last_active":"2024-09-05T02:02:27.207776+0000","last_peered":"2024-09-05T02:02:27.207776+0000","last_clean":"2024-09-05T02:02:27.207776+0000","last_became_active":"2024-09-05T02:00:14.366287+0000","last_became_peered":"2024-09-05T02:00:14.366287+0000","last_unstale":"2024-09-05T02:02:27.207776+0000","last_undegraded":"2024-09-05T02:02:27.207776+0000","last_fullsized":"2024-09-05T02:02:27.207776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:36:57.143489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207640+0000","last_change":"2024-09-05T02:00:14.360311+0000","last_active":"2024-09-05T02:02:27.207640+0000","last_peered":"2024-09-05T02:02:27.207640+0000","last_clean":"2024-09-05T02:02:27.207640+0000","last_became_active":"2024-09-05T02:00:14.360028+0000","last_became_peered":"2024-09-05T02:00:14.360028+0000","last_unstale":"2024-09-05T02:02:27.207640+0000","last_undegraded":"2024-09-05T02:02:27.207640+0000","last_fullsized":"2024-09-05T02:02:27.207640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:35:38.804555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207867+0000","last_change":"2024-09-05T02:00:14.365853+0000","last_active":"2024-09-05T02:02:27.207867+0000","last_peered":"2024-09-05T02:02:27.207867+0000","last_clean":"2024-09-05T02:02:27.207867+0000","last_became_active":"2024-09-05T02:00:14.365549+0000","last_became_peered":"2024-09-05T02:00:14.365549+0000","last_unstale":"2024-09-05T02:02:27.207867+0000","last_undegraded":"2024-09-05T02:02:27.207867+0000","last_fullsized":"2024-09-05T02:02:27.207867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:58:45.897592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200763+0000","last_change":"2024-09-05T02:00:14.367357+0000","last_active":"2024-09-05T02:02:27.200763+0000","last_peered":"2024-09-05T02:02:27.200763+0000","last_clean":"2024-09-05T02:02:27.200763+0000","last_became_active":"2024-09-05T02:00:14.366794+0000","last_became_peered":"2024-09-05T02:00:14.366794+0000","last_unstale":"2024-09-05T02:02:27.200763+0000","last_undegraded":"2024-09-05T02:02:27.200763+0000","last_fullsized":"2024-09-05T02:02:27.200763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:40:51.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359605+0000","last_change":"2024-09-05T02:00:14.356991+0000","last_active":"2024-09-05T02:02:27.359605+0000","last_peered":"2024-09-05T02:02:27.359605+0000","last_clean":"2024-09-05T02:02:27.359605+0000","last_became_active":"2024-09-05T02:00:14.356662+0000","last_became_peered":"2024-09-05T02:00:14.356662+0000","last_unstale":"2024-09-05T02:02:27.359605+0000","last_undegraded":"2024-09-05T02:02:27.359605+0000","last_fullsized":"2024-09-05T02:02:27.359605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:39:53.087448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205123+0000","last_change":"2024-09-05T02:00:14.357750+0000","last_active":"2024-09-05T02:02:27.205123+0000","last_peered":"2024-09-05T02:02:27.205123+0000","last_clean":"2024-09-05T02:02:27.205123+0000","last_became_active":"2024-09-05T02:00:14.357413+0000","last_became_peered":"2024-09-05T02:00:14.357413+0000","last_unstale":"2024-09-05T02:02:27.205123+0000","last_undegraded":"2024-09-05T02:02:27.205123+0000","last_fullsized":"2024-09-05T02:02:27.205123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:29:47.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.204535+0000","last_change":"2024-09-05T02:00:14.364068+0000","last_active":"2024-09-05T02:02:27.204535+0000","last_peered":"2024-09-05T02:02:27.204535+0000","last_clean":"2024-09-05T02:02:27.204535+0000","last_became_active":"2024-09-05T02:00:14.363674+0000","last_became_peered":"2024-09-05T02:00:14.363674+0000","last_unstale":"2024-09-05T02:02:27.204535+0000","last_undegraded":"2024-09-05T02:02:27.204535+0000","last_fullsized":"2024-09-05T02:02:27.204535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:28:00.788668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359585+0000","last_change":"2024-09-05T02:00:14.355152+0000","last_active":"2024-09-05T02:02:27.359585+0000","last_peered":"2024-09-05T02:02:27.359585+0000","last_clean":"2024-09-05T02:02:27.359585+0000","last_became_active":"2024-09-05T02:00:14.354874+0000","last_became_peered":"2024-09-05T02:00:14.354874+0000","last_unstale":"2024-09-05T02:02:27.359585+0000","last_undegraded":"2024-09-05T02:02:27.359585+0000","last_fullsized":"2024-09-05T02:02:27.359585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:14:17.069049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.206078+0000","last_change":"2024-09-05T02:00:14.360241+0000","last_active":"2024-09-05T02:02:27.206078+0000","last_peered":"2024-09-05T02:02:27.206078+0000","last_clean":"2024-09-05T02:02:27.206078+0000","last_became_active":"2024-09-05T02:00:14.359648+0000","last_became_peered":"2024-09-05T02:00:14.359648+0000","last_unstale":"2024-09-05T02:02:27.206078+0000","last_undegraded":"2024-09-05T02:02:27.206078+0000","last_fullsized":"2024-09-05T02:02:27.206078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:15.060575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200191+0000","last_change":"2024-09-05T02:00:14.366094+0000","last_active":"2024-09-05T02:02:27.200191+0000","last_peered":"2024-09-05T02:02:27.200191+0000","last_clean":"2024-09-05T02:02:27.200191+0000","last_became_active":"2024-09-05T02:00:14.365788+0000","last_became_peered":"2024-09-05T02:00:14.365788+0000","last_unstale":"2024-09-05T02:02:27.200191+0000","last_undegraded":"2024-09-05T02:02:27.200191+0000","last_fullsized":"2024-09-05T02:02:27.200191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:23:00.909487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199671+0000","last_change":"2024-09-05T02:00:14.364651+0000","last_active":"2024-09-05T02:02:27.199671+0000","last_peered":"2024-09-05T02:02:27.199671+0000","last_clean":"2024-09-05T02:02:27.199671+0000","last_became_active":"2024-09-05T02:00:14.364365+0000","last_became_peered":"2024-09-05T02:00:14.364365+0000","last_unstale":"2024-09-05T02:02:27.199671+0000","last_undegraded":"2024-09-05T02:02:27.199671+0000","last_fullsized":"2024-09-05T02:02:27.199671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:10:14.978382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199677+0000","last_change":"2024-09-05T02:00:14.364761+0000","last_active":"2024-09-05T02:02:27.199677+0000","last_peered":"2024-09-05T02:02:27.199677+0000","last_clean":"2024-09-05T02:02:27.199677+0000","last_became_active":"2024-09-05T02:00:14.364204+0000","last_became_peered":"2024-09-05T02:00:14.364204+0000","last_unstale":"2024-09-05T02:02:27.199677+0000","last_undegraded":"2024-09-05T02:02:27.199677+0000","last_fullsized":"2024-09-05T02:02:27.199677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:16:51.076304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.359554+0000","last_change":"2024-09-05T02:00:14.357632+0000","last_active":"2024-09-05T02:02:27.359554+0000","last_peered":"2024-09-05T02:02:27.359554+0000","last_clean":"2024-09-05T02:02:27.359554+0000","last_became_active":"2024-09-05T02:00:14.357348+0000","last_became_peered":"2024-09-05T02:00:14.357348+0000","last_unstale":"2024-09-05T02:02:27.359554+0000","last_undegraded":"2024-09-05T02:02:27.359554+0000","last_fullsized":"2024-09-05T02:02:27.359554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:11.163292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362670+0000","last_change":"2024-09-05T02:00:14.361484+0000","last_active":"2024-09-05T02:02:27.362670+0000","last_peered":"2024-09-05T02:02:27.362670+0000","last_clean":"2024-09-05T02:02:27.362670+0000","last_became_active":"2024-09-05T02:00:14.361156+0000","last_became_peered":"2024-09-05T02:00:14.361156+0000","last_unstale":"2024-09-05T02:02:27.362670+0000","last_undegraded":"2024-09-05T02:02:27.362670+0000","last_fullsized":"2024-09-05T02:02:27.362670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:40.124733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203916+0000","last_change":"2024-09-05T02:00:14.358325+0000","last_active":"2024-09-05T02:02:27.203916+0000","last_peered":"2024-09-05T02:02:27.203916+0000","last_clean":"2024-09-05T02:02:27.203916+0000","last_became_active":"2024-09-05T02:00:14.357993+0000","last_became_peered":"2024-09-05T02:00:14.357993+0000","last_unstale":"2024-09-05T02:02:27.203916+0000","last_undegraded":"2024-09-05T02:02:27.203916+0000","last_fullsized":"2024-09-05T02:02:27.203916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:18.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.362551+0000","last_change":"2024-09-05T02:00:14.365291+0000","last_active":"2024-09-05T02:02:27.362551+0000","last_peered":"2024-09-05T02:02:27.362551+0000","last_clean":"2024-09-05T02:02:27.362551+0000","last_became_active":"2024-09-05T02:00:14.364937+0000","last_became_peered":"2024-09-05T02:00:14.364937+0000","last_unstale":"2024-09-05T02:02:27.362551+0000","last_undegraded":"2024-09-05T02:02:27.362551+0000","last_fullsized":"2024-09-05T02:02:27.362551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:42:57.712651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.200788+0000","last_change":"2024-09-05T02:00:14.367603+0000","last_active":"2024-09-05T02:02:27.200788+0000","last_peered":"2024-09-05T02:02:27.200788+0000","last_clean":"2024-09-05T02:02:27.200788+0000","last_became_active":"2024-09-05T02:00:14.367267+0000","last_became_peered":"2024-09-05T02:00:14.367267+0000","last_unstale":"2024-09-05T02:02:27.200788+0000","last_undegraded":"2024-09-05T02:02:27.200788+0000","last_fullsized":"2024-09-05T02:02:27.200788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:51:34.595898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.207492+0000","last_change":"2024-09-05T02:00:14.369499+0000","last_active":"2024-09-05T02:02:27.207492+0000","last_peered":"2024-09-05T02:02:27.207492+0000","last_clean":"2024-09-05T02:02:27.207492+0000","last_became_active":"2024-09-05T02:00:14.369169+0000","last_became_peered":"2024-09-05T02:00:14.369169+0000","last_unstale":"2024-09-05T02:02:27.207492+0000","last_undegraded":"2024-09-05T02:02:27.207492+0000","last_fullsized":"2024-09-05T02:02:27.207492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:17.161204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.203441+0000","last_change":"2024-09-05T02:00:14.360320+0000","last_active":"2024-09-05T02:02:27.203441+0000","last_peered":"2024-09-05T02:02:27.203441+0000","last_clean":"2024-09-05T02:02:27.203441+0000","last_became_active":"2024-09-05T02:00:14.359991+0000","last_became_peered":"2024-09-05T02:00:14.359991+0000","last_unstale":"2024-09-05T02:02:27.203441+0000","last_undegraded":"2024-09-05T02:02:27.203441+0000","last_fullsized":"2024-09-05T02:02:27.203441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:47:03.721863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.361412+0000","last_change":"2024-09-05T02:00:14.364660+0000","last_active":"2024-09-05T02:02:27.361412+0000","last_peered":"2024-09-05T02:02:27.361412+0000","last_clean":"2024-09-05T02:02:27.361412+0000","last_became_active":"2024-09-05T02:00:14.364313+0000","last_became_peered":"2024-09-05T02:00:14.364313+0000","last_unstale":"2024-09-05T02:02:27.361412+0000","last_undegraded":"2024-09-05T02:02:27.361412+0000","last_fullsized":"2024-09-05T02:02:27.361412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:13:47.496063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.208875+0000","last_change":"2024-09-05T02:00:14.359853+0000","last_active":"2024-09-05T02:02:27.208875+0000","last_peered":"2024-09-05T02:02:27.208875+0000","last_clean":"2024-09-05T02:02:27.208875+0000","last_became_active":"2024-09-05T02:00:14.359543+0000","last_became_peered":"2024-09-05T02:00:14.359543+0000","last_unstale":"2024-09-05T02:02:27.208875+0000","last_undegraded":"2024-09-05T02:02:27.208875+0000","last_fullsized":"2024-09-05T02:02:27.208875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:30:29.415492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.358876+0000","last_change":"2024-09-05T02:01:26.270888+0000","last_active":"2024-09-05T02:02:27.358876+0000","last_peered":"2024-09-05T02:02:27.358876+0000","last_clean":"2024-09-05T02:02:27.358876+0000","last_became_active":"2024-09-05T02:01:26.270607+0000","last_became_peered":"2024-09-05T02:01:26.270607+0000","last_unstale":"2024-09-05T02:02:27.358876+0000","last_undegraded":"2024-09-05T02:02:27.358876+0000","last_fullsized":"2024-09-05T02:02:27.358876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:15:16.563521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199910+0000","last_change":"2024-09-05T02:00:14.367191+0000","last_active":"2024-09-05T02:02:27.199910+0000","last_peered":"2024-09-05T02:02:27.199910+0000","last_clean":"2024-09-05T02:02:27.199910+0000","last_became_active":"2024-09-05T02:00:14.366831+0000","last_became_peered":"2024-09-05T02:00:14.366831+0000","last_unstale":"2024-09-05T02:02:27.199910+0000","last_undegraded":"2024-09-05T02:02:27.199910+0000","last_fullsized":"2024-09-05T02:02:27.199910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:55.690642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.199853+0000","last_change":"2024-09-05T02:00:14.355774+0000","last_active":"2024-09-05T02:02:27.199853+0000","last_peered":"2024-09-05T02:02:27.199853+0000","last_clean":"2024-09-05T02:02:27.199853+0000","last_became_active":"2024-09-05T02:00:14.355293+0000","last_became_peered":"2024-09-05T02:00:14.355293+0000","last_unstale":"2024-09-05T02:02:27.199853+0000","last_undegraded":"2024-09-05T02:02:27.199853+0000","last_fullsized":"2024-09-05T02:02:27.199853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:07:00.547820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T02:02:27.205824+0000","last_change":"2024-09-05T02:00:14.359978+0000","last_active":"2024-09-05T02:02:27.205824+0000","last_peered":"2024-09-05T02:02:27.205824+0000","last_clean":"2024-09-05T02:02:27.205824+0000","last_became_active":"2024-09-05T02:00:14.359209+0000","last_became_peered":"2024-09-05T02:00:14.359209+0000","last_unstale":"2024-09-05T02:02:27.205824+0000","last_undegraded":"2024-09-05T02:02:27.205824+0000","last_fullsized":"2024-09-05T02:02:27.205824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T02:00:13.336966+0000","last_clean_scrub_stamp":"2024-09-05T02:00:13.336966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:27.848906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76109,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1090,"num_read_kb":626,"num_write":272,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":224508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":516663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808881,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1740800,"data_stored":1170574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":520155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201340,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":516437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430271,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1114562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593798,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":512058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822729,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36980,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708172,"statfs":{"total":95995035648,"available":95957168128,"internally_reserved":0,"allocated":1667072,"data_stored":1108659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986256,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":516244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1052672,"data_stored":511271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444121,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":581893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":575908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":514048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T02:02:46.356 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:02:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:46 smithi167 ceph-mon[35122]: pgmap v360: 129 pgs: 129 active+clean; 651 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-05T02:02:46.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:46 smithi139 ceph-mon[34262]: pgmap v360: 129 pgs: 129 active+clean; 651 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-05T02:02:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:46 smithi103 ceph-mon[30902]: pgmap v360: 129 pgs: 129 active+clean; 651 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-05T02:02:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.088+0000 7f03ab929640 1 -- 172.21.15.103:0/1034197395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4101420 msgr2=0x7f03a410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.088+0000 7f03ab929640 1 --2- 172.21.15.103:0/1034197395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4101420 0x7f03a410d950 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f039c009cd0 tx=0x7f039c02a8e0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.088+0000 7f03ab929640 1 -- 172.21.15.103:0/1034197395 shutdown_connections 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.088+0000 7f03ab929640 1 --2- 172.21.15.103:0/1034197395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4101420 0x7f03a410d950 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.088+0000 7f03ab929640 1 --2- 172.21.15.103:0/1034197395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a4100a80 0x7f03a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.088+0000 7f03ab929640 1 --2- 172.21.15.103:0/1034197395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a4106a80 0x7f03a4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.088+0000 7f03ab929640 1 -- 172.21.15.103:0/1034197395 >> 172.21.15.103:0/1034197395 conn(0x7f03a40fc7a0 msgr2=0x7f03a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.088+0000 7f03ab929640 1 -- 172.21.15.103:0/1034197395 shutdown_connections 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.088+0000 7f03ab929640 1 -- 172.21.15.103:0/1034197395 wait complete. 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03ab929640 1 Processor -- start 2024-09-05T02:02:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03ab929640 1 -- start start 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03ab929640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4100a80 0x7f03a41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03ab929640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a4101420 0x7f03a4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03ab929640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a4106a80 0x7f03a419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03ab929640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4110d30 con 0x7f03a4100a80 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03ab929640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4110bb0 con 0x7f03a4101420 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03ab929640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4110eb0 con 0x7f03a4106a80 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03a96c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4100a80 0x7f03a41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03a96c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4100a80 0x7f03a41982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55640/0 (socket says 172.21.15.103:55640) 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.089+0000 7f03a96c5640 1 -- 172.21.15.103:0/2756863355 learned_addr learned my addr 172.21.15.103:0/2756863355 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03a8ec4640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a4101420 0x7f03a4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03a9ec6640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a4106a80 0x7f03a419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03a96c5640 1 -- 172.21.15.103:0/2756863355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a4106a80 msgr2=0x7f03a419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03a96c5640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a4106a80 0x7f03a419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03a96c5640 1 -- 172.21.15.103:0/2756863355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a4101420 msgr2=0x7f03a4198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03a96c5640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a4101420 0x7f03a4198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03a96c5640 1 -- 172.21.15.103:0/2756863355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a419d270 con 0x7f03a4100a80 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03a96c5640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4100a80 0x7f03a41982c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f03a000e9f0 tx=0x7f03a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f039a7fc640 1 -- 172.21.15.103:0/2756863355 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a000cd80 con 0x7f03a4100a80 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f039a7fc640 1 -- 172.21.15.103:0/2756863355 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03a00054d0 con 0x7f03a4100a80 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a419d560 con 0x7f03a4100a80 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f039a7fc640 1 -- 172.21.15.103:0/2756863355 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0010490 con 0x7f03a4100a80 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.090+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a41a4e40 con 0x7f03a4100a80 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.091+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a4100ee0 con 0x7f03a4100a80 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.092+0000 7f039a7fc640 1 -- 172.21.15.103:0/2756863355 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f03a0005000 con 0x7f03a4100a80 2024-09-05T02:02:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.093+0000 7f039a7fc640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03900a2b80 0x7f03900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.093+0000 7f03a8ec4640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03900a2b80 0x7f03900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.093+0000 7f039a7fc640 1 -- 172.21.15.103:0/2756863355 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f03a00cfab0 con 0x7f03a4100a80 2024-09-05T02:02:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.093+0000 7f03a8ec4640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03900a2b80 0x7f03900a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0394005fd0 tx=0x7f0394005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.096+0000 7f039a7fc640 1 -- 172.21.15.103:0/2756863355 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03a00541b0 con 0x7f03a4100a80 2024-09-05T02:02:47.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.298+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03a410a060 con 0x7f03a4100a80 2024-09-05T02:02:47.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:47 smithi103 ceph-mon[30902]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:02:47.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.299+0000 7f039a7fc640 1 -- 172.21.15.103:0/2756863355 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f03a00052f0 con 0x7f03a4100a80 2024-09-05T02:02:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:47.301 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":29,"btime":"2024-09-05T02:02:41:223802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:02:41.148036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:02:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 29 2024-09-05T02:02:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03900a2b80 msgr2=0x7f03900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03900a2b80 0x7f03900a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0394005fd0 tx=0x7f0394005850 comp rx=0 tx=0).stop 2024-09-05T02:02:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4100a80 msgr2=0x7f03a41982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4100a80 0x7f03a41982c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f03a000e9f0 tx=0x7f03a000edf0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 shutdown_connections 2024-09-05T02:02:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f03900a2b80 0x7f03900a5040 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a4106a80 0x7f03a419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a4101420 0x7f03a4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 --2- 172.21.15.103:0/2756863355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a4100a80 0x7f03a41982c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.301+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 >> 172.21.15.103:0/2756863355 conn(0x7f03a40fc7a0 msgr2=0x7f03a406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.302+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 shutdown_connections 2024-09-05T02:02:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:47.302+0000 7f03ab929640 1 -- 172.21.15.103:0/2756863355 wait complete. 2024-09-05T02:02:47.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:47 smithi167 ceph-mon[35122]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:02:47.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:47 smithi139 ceph-mon[34262]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T02:02:48.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:02:48.257 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:48.377 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:48 smithi103 ceph-mon[30902]: pgmap v361: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-05T02:02:48.377 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:48 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2756863355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:48 smithi167 ceph-mon[35122]: pgmap v361: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-05T02:02:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:48 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2756863355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:48.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:48 smithi139 ceph-mon[34262]: pgmap v361: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-05T02:02:48.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:48 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2756863355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:48.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 -- 172.21.15.103:0/484913132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0100a80 msgr2=0x7f6dd0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:48.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 --2- 172.21.15.103:0/484913132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0100a80 0x7f6dd0100ee0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009930 tx=0x7f6db802a420 comp rx=0 tx=0).stop 2024-09-05T02:02:48.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 -- 172.21.15.103:0/484913132 shutdown_connections 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 --2- 172.21.15.103:0/484913132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dd0101420 0x7f6dd010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 --2- 172.21.15.103:0/484913132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0100a80 0x7f6dd0100ee0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 --2- 172.21.15.103:0/484913132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dd0106a80 0x7f6dd0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 -- 172.21.15.103:0/484913132 >> 172.21.15.103:0/484913132 conn(0x7f6dd00fc7a0 msgr2=0x7f6dd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 -- 172.21.15.103:0/484913132 shutdown_connections 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 -- 172.21.15.103:0/484913132 wait complete. 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 Processor -- start 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.714+0000 7f6dd6834640 1 -- start start 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd6834640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dd0100a80 0x7f6dd0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd6834640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dd0101420 0x7f6dd0198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd6834640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0106a80 0x7f6dd019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd6834640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0110d70 con 0x7f6dd0106a80 2024-09-05T02:02:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd6834640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0110bf0 con 0x7f6dd0101420 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd6834640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0110ef0 con 0x7f6dd0100a80 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd4dd1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0106a80 0x7f6dd019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd4dd1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0106a80 0x7f6dd019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55654/0 (socket says 172.21.15.103:55654) 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd4dd1640 1 -- 172.21.15.103:0/2307786244 learned_addr learned my addr 172.21.15.103:0/2307786244 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dcf7fe640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dd0101420 0x7f6dd0198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dcffff640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dd0100a80 0x7f6dd0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd4dd1640 1 -- 172.21.15.103:0/2307786244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dd0100a80 msgr2=0x7f6dd0198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd4dd1640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dd0100a80 0x7f6dd0198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd4dd1640 1 -- 172.21.15.103:0/2307786244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dd0101420 msgr2=0x7f6dd0198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd4dd1640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dd0101420 0x7f6dd0198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd4dd1640 1 -- 172.21.15.103:0/2307786244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd019d400 con 0x7f6dd0106a80 2024-09-05T02:02:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dd4dd1640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0106a80 0x7f6dd019cd20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800b970 tx=0x7f6dc800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/2307786244 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8019070 con 0x7f6dd0106a80 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.715+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/2307786244 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dc8014440 con 0x7f6dd0106a80 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.716+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd019d6f0 con 0x7f6dd0106a80 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.716+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/2307786244 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc80075d0 con 0x7f6dd0106a80 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.716+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd01a4fd0 con 0x7f6dd0106a80 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.717+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd0100ee0 con 0x7f6dd0106a80 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.718+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/2307786244 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6dc801c070 con 0x7f6dd0106a80 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.718+0000 7f6dcd7fa640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6dbc0a2b80 0x7f6dbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.719+0000 7f6dcffff640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6dbc0a2b80 0x7f6dbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.719+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/2307786244 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6dc80ce9b0 con 0x7f6dd0106a80 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.719+0000 7f6dcffff640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6dbc0a2b80 0x7f6dbc0a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005fd0 tx=0x7f6dc000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.723+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/2307786244 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6dc809af10 con 0x7f6dd0106a80 2024-09-05T02:02:48.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.924+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dd0109a90 con 0x7f6dd0106a80 2024-09-05T02:02:48.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.925+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/2307786244 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f6dc809f210 con 0x7f6dd0106a80 2024-09-05T02:02:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:48.927 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":29,"btime":"2024-09-05T02:02:41:223802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:02:41.148036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:02:48.928 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 29 2024-09-05T02:02:48.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6dbc0a2b80 msgr2=0x7f6dbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:48.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6dbc0a2b80 0x7f6dbc0a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005fd0 tx=0x7f6dc000b390 comp rx=0 tx=0).stop 2024-09-05T02:02:48.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0106a80 msgr2=0x7f6dd019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:48.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0106a80 0x7f6dd019cd20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800b970 tx=0x7f6dc800be30 comp rx=0 tx=0).stop 2024-09-05T02:02:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 shutdown_connections 2024-09-05T02:02:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd0106a80 0x7f6dd019cd20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dd0101420 0x7f6dd0198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6dbc0a2b80 0x7f6dbc0a5040 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 --2- 172.21.15.103:0/2307786244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dd0100a80 0x7f6dd0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 >> 172.21.15.103:0/2307786244 conn(0x7f6dd00fc7a0 msgr2=0x7f6dd006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 shutdown_connections 2024-09-05T02:02:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:48.927+0000 7f6dd6834640 1 -- 172.21.15.103:0/2307786244 wait complete. 2024-09-05T02:02:49.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:49 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2307786244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:49.603 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:02:49.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:49 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2307786244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:49.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:49 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2307786244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:49.851 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.483+0000 7fe05b9c7640 1 -- 172.21.15.103:0/2056103607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054100e20 msgr2=0x7fe054101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.483+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/2056103607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054100e20 0x7fe054101200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe044009900 tx=0x7fe04402a480 comp rx=0 tx=0).stop 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.483+0000 7fe05b9c7640 1 -- 172.21.15.103:0/2056103607 shutdown_connections 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.483+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/2056103607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe054102ef0 0x7fe05410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.483+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/2056103607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe054102570 0x7fe0541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.483+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/2056103607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054100e20 0x7fe054101200 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.483+0000 7fe05b9c7640 1 -- 172.21.15.103:0/2056103607 >> 172.21.15.103:0/2056103607 conn(0x7fe0540fc7d0 msgr2=0x7fe0540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.483+0000 7fe05b9c7640 1 -- 172.21.15.103:0/2056103607 shutdown_connections 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.483+0000 7fe05b9c7640 1 -- 172.21.15.103:0/2056103607 wait complete. 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe05b9c7640 1 Processor -- start 2024-09-05T02:02:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe05b9c7640 1 -- start start 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe05b9c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe054100e20 0x7fe05419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe05b9c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054102570 0x7fe05419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe05b9c7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe054102ef0 0x7fe05419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe05b9c7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe054110e30 con 0x7fe054102570 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe05b9c7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe054110cb0 con 0x7fe054100e20 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe05b9c7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe054110fb0 con 0x7fe054102ef0 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe058f62640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054102570 0x7fe05419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe058f62640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054102570 0x7fe05419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60964/0 (socket says 172.21.15.103:60964) 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe058f62640 1 -- 172.21.15.103:0/358176679 learned_addr learned my addr 172.21.15.103:0/358176679 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe059763640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe054100e20 0x7fe05419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe059f64640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe054102ef0 0x7fe05419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe058f62640 1 -- 172.21.15.103:0/358176679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe054102ef0 msgr2=0x7fe05419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe058f62640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe054102ef0 0x7fe05419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe058f62640 1 -- 172.21.15.103:0/358176679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe054100e20 msgr2=0x7fe05419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe058f62640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe054100e20 0x7fe05419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe058f62640 1 -- 172.21.15.103:0/358176679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05419f5b0 con 0x7fe054102570 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe059f64640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe054102ef0 0x7fe05419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.484+0000 7fe058f62640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054102570 0x7fe05419ab40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe05000d950 tx=0x7fe05000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.485+0000 7fe04a7fc640 1 -- 172.21.15.103:0/358176679 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050017070 con 0x7fe054102570 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.485+0000 7fe04a7fc640 1 -- 172.21.15.103:0/358176679 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe050009c30 con 0x7fe054102570 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.485+0000 7fe04a7fc640 1 -- 172.21.15.103:0/358176679 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05000f620 con 0x7fe054102570 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.485+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05419f8a0 con 0x7fe054102570 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.485+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0541a7180 con 0x7fe054102570 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.486+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe054101200 con 0x7fe054102570 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.487+0000 7fe04a7fc640 1 -- 172.21.15.103:0/358176679 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe0500109a0 con 0x7fe054102570 2024-09-05T02:02:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.487+0000 7fe04a7fc640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0400a2b80 0x7fe0400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:50.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.488+0000 7fe059763640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0400a2b80 0x7fe0400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:50.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.488+0000 7fe04a7fc640 1 -- 172.21.15.103:0/358176679 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe0500ce960 con 0x7fe054102570 2024-09-05T02:02:50.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.488+0000 7fe059763640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0400a2b80 0x7fe0400a5040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe044009230 tx=0x7fe0440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:50.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.492+0000 7fe04a7fc640 1 -- 172.21.15.103:0/358176679 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe050009de0 con 0x7fe054102570 2024-09-05T02:02:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:50 smithi167 ceph-mon[35122]: pgmap v362: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-05T02:02:50.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:50 smithi139 ceph-mon[34262]: pgmap v362: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-05T02:02:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:50 smithi103 ceph-mon[30902]: pgmap v362: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-05T02:02:50.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.661+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0541a7350 con 0x7fe054102570 2024-09-05T02:02:50.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.663+0000 7fe04a7fc640 1 -- 172.21.15.103:0/358176679 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24926 (secure 0 0 0) 0x7fe05009b510 con 0x7fe054102570 2024-09-05T02:02:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:50.665 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":90,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:02:27.192641+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0400a2b80 msgr2=0x7fe0400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0400a2b80 0x7fe0400a5040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe044009230 tx=0x7fe0440058d0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054102570 msgr2=0x7fe05419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054102570 0x7fe05419ab40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe05000d950 tx=0x7fe05000de10 comp rx=0 tx=0).stop 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 shutdown_connections 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0400a2b80 0x7fe0400a5040 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe054102ef0 0x7fe05419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe054102570 0x7fe05419ab40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 --2- 172.21.15.103:0/358176679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe054100e20 0x7fe05419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 >> 172.21.15.103:0/358176679 conn(0x7fe0540fc7d0 msgr2=0x7fe05406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 shutdown_connections 2024-09-05T02:02:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:50.665+0000 7fe05b9c7640 1 -- 172.21.15.103:0/358176679 wait complete. 2024-09-05T02:02:51.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-05T02:02:51.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:51 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/358176679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:02:51.640 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:51.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:51 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/358176679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:02:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:51 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/358176679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:02:52.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:52 smithi103 ceph-mon[30902]: pgmap v363: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-05T02:02:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:52 smithi167 ceph-mon[35122]: pgmap v363: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-05T02:02:52.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:52 smithi139 ceph-mon[34262]: pgmap v363: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-05T02:02:52.968 INFO:teuthology.orchestra.run.smithi103.stdout:{"run_dir":"/var/run/ceph"} 2024-09-05T02:02:52.968 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-05T02:02:52.969 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1743 2024-09-05T02:02:52.969 DEBUG:tasks.quiescer:Yielding 2024-09-05T02:02:52.969 INFO:teuthology.run_tasks:Running task workunit... 2024-09-05T02:02:52.984 INFO:tasks.workunit:Pulling workunits from ref 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T02:02:52.984 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-05T02:02:52.984 INFO:tasks.workunit:timeout=3h 2024-09-05T02:02:52.984 INFO:tasks.workunit:cleanup=True 2024-09-05T02:02:52.984 DEBUG:teuthology.orchestra.run.smithi103:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:52.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:02:53.002 INFO:teuthology.orchestra.run.smithi103.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:53.002 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-05T02:02:53.002 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6dh/109d Inode: 1 Links: 2 2024-09-05T02:02:53.002 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T02:02:53.002 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-05T02:02:53.002 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-05 02:01:02.604631069 +0000 2024-09-05T02:02:53.002 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-05 02:01:02.604631069 +0000 2024-09-05T02:02:53.002 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-05 02:02:40.790966181 +0000 2024-09-05T02:02:53.002 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2024-09-05T02:02:53.003 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-05T02:02:53.003 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-05T02:02:53.073 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T02:02:53.130 INFO:tasks.workunit.client.0.smithi103.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-05T02:02:53.248 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.749+0000 7fe92da5b640 1 -- 172.21.15.103:0/2398651476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 msgr2=0x7fe9281067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.749+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2398651476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 0x7fe9281067f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe910009900 tx=0x7fe91002a410 comp rx=0 tx=0).stop 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.749+0000 7fe92da5b640 1 -- 172.21.15.103:0/2398651476 shutdown_connections 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.749+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2398651476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe928106f20 0x7fe92810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.749+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2398651476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 0x7fe9281067f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.749+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2398651476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe928101df0 0x7fe9281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.749+0000 7fe92da5b640 1 -- 172.21.15.103:0/2398651476 >> 172.21.15.103:0/2398651476 conn(0x7fe9280fd8b0 msgr2=0x7fe9280ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.749+0000 7fe92da5b640 1 -- 172.21.15.103:0/2398651476 shutdown_connections 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.749+0000 7fe92da5b640 1 -- 172.21.15.103:0/2398651476 wait complete. 2024-09-05T02:02:53.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.750+0000 7fe92da5b640 1 Processor -- start 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.750+0000 7fe92da5b640 1 -- start start 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.750+0000 7fe92da5b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe928101df0 0x7fe9280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.750+0000 7fe92da5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 0x7fe928074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe92da5b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe928106f20 0x7fe928074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe92da5b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe928111e30 con 0x7fe9281027a0 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe92da5b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe928111cb0 con 0x7fe928106f20 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe92da5b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe928111fb0 con 0x7fe928101df0 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe91e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 0x7fe928074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe91e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 0x7fe928074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60992/0 (socket says 172.21.15.103:60992) 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe91e7fc640 1 -- 172.21.15.103:0/2243934584 learned_addr learned my addr 172.21.15.103:0/2243934584 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:53.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe91e7fc640 1 -- 172.21.15.103:0/2243934584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe928101df0 msgr2=0x7fe9280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe926ffd640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe928101df0 0x7fe9280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe9277fe640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe928106f20 0x7fe928074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe91e7fc640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe928101df0 0x7fe9280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe91e7fc640 1 -- 172.21.15.103:0/2243934584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe928106f20 msgr2=0x7fe928074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe91e7fc640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe928106f20 0x7fe928074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe91e7fc640 1 -- 172.21.15.103:0/2243934584 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9281a6850 con 0x7fe9281027a0 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe926ffd640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe928101df0 0x7fe9280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe91e7fc640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 0x7fe928074a30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe910005940 tx=0x7fe9100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe92ca59640 1 -- 172.21.15.103:0/2243934584 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91003f030 con 0x7fe9281027a0 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9281a6ae0 con 0x7fe9281027a0 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe92ca59640 1 -- 172.21.15.103:0/2243934584 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe910003cb0 con 0x7fe9281027a0 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe92ca59640 1 -- 172.21.15.103:0/2243934584 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910035070 con 0x7fe9281027a0 2024-09-05T02:02:53.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.751+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9281a6fc0 con 0x7fe9281027a0 2024-09-05T02:02:53.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.753+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9281021d0 con 0x7fe9281027a0 2024-09-05T02:02:53.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.753+0000 7fe92ca59640 1 -- 172.21.15.103:0/2243934584 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe910042030 con 0x7fe9281027a0 2024-09-05T02:02:53.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.754+0000 7fe92ca59640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9140a2b80 0x7fe9140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:53.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.754+0000 7fe926ffd640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9140a2b80 0x7fe9140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:53.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.754+0000 7fe92ca59640 1 -- 172.21.15.103:0/2243934584 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe9100efcb0 con 0x7fe9281027a0 2024-09-05T02:02:53.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.755+0000 7fe926ffd640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9140a2b80 0x7fe9140a5040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe918009b50 tx=0x7fe918005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:53.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.758+0000 7fe92ca59640 1 -- 172.21.15.103:0/2243934584 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9100bc860 con 0x7fe9281027a0 2024-09-05T02:02:53.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.979+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe92810a160 con 0x7fe9281027a0 2024-09-05T02:02:53.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.979+0000 7fe92ca59640 1 -- 172.21.15.103:0/2243934584 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7fe9100bc160 con 0x7fe9281027a0 2024-09-05T02:02:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:02:53.982 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":29,"btime":"2024-09-05T02:02:41:223802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:02:41.148036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:02:53.982 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 29 2024-09-05T02:02:53.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9140a2b80 msgr2=0x7fe9140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9140a2b80 0x7fe9140a5040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe918009b50 tx=0x7fe918005f90 comp rx=0 tx=0).stop 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 msgr2=0x7fe928074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 0x7fe928074a30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe910005940 tx=0x7fe9100042f0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 shutdown_connections 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe928106f20 0x7fe928074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9281027a0 0x7fe928074a30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe9140a2b80 0x7fe9140a5040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 --2- 172.21.15.103:0/2243934584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe928101df0 0x7fe9280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 >> 172.21.15.103:0/2243934584 conn(0x7fe9280fd8b0 msgr2=0x7fe9280fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 shutdown_connections 2024-09-05T02:02:53.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:53.982+0000 7fe92da5b640 1 -- 172.21.15.103:0/2243934584 wait complete. 2024-09-05T02:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2145867746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc100a60 msgr2=0x7fd3dc100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2145867746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc100a60 0x7fd3dc100e60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009900 tx=0x7fd3d802a410 comp rx=0 tx=0).stop 2024-09-05T02:02:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2145867746 shutdown_connections 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2145867746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3dc102610 0x7fd3dc108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2145867746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3dc101c50 0x7fd3dc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2145867746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc100a60 0x7fd3dc100e60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2145867746 >> 172.21.15.103:0/2145867746 conn(0x7fd3dc0fc1e0 msgr2=0x7fd3dc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2145867746 shutdown_connections 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2145867746 wait complete. 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 Processor -- start 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.415+0000 7fd3e42e6640 1 -- start start 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e42e6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3dc100a60 0x7fd3dc1049f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e42e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc101c50 0x7fd3dc103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e42e6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3dc102610 0x7fd3dc103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e42e6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc10c5f0 con 0x7fd3dc101c50 2024-09-05T02:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e42e6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc10c470 con 0x7fd3dc100a60 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e42e6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc10c770 con 0x7fd3dc102610 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e1881640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc101c50 0x7fd3dc103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e1881640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc101c50 0x7fd3dc103030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32782/0 (socket says 172.21.15.103:32782) 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e1881640 1 -- 172.21.15.103:0/2299155687 learned_addr learned my addr 172.21.15.103:0/2299155687 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e1881640 1 -- 172.21.15.103:0/2299155687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3dc102610 msgr2=0x7fd3dc103570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e1881640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3dc102610 0x7fd3dc103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e1881640 1 -- 172.21.15.103:0/2299155687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3dc100a60 msgr2=0x7fd3dc1049f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e2082640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3dc100a60 0x7fd3dc1049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e1881640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3dc100a60 0x7fd3dc1049f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e1881640 1 -- 172.21.15.103:0/2299155687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc103e00 con 0x7fd3dc101c50 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e1881640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc101c50 0x7fd3dc103030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00c890 tx=0x7fd3cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3d37fe640 1 -- 172.21.15.103:0/2299155687 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc004220 con 0x7fd3dc101c50 2024-09-05T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3d37fe640 1 -- 172.21.15.103:0/2299155687 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3cc0043c0 con 0x7fd3dc101c50 2024-09-05T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2299155687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc1a1ef0 con 0x7fd3dc101c50 2024-09-05T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3d37fe640 1 -- 172.21.15.103:0/2299155687 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc004a00 con 0x7fd3dc101c50 2024-09-05T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.416+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2299155687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc1a23d0 con 0x7fd3dc101c50 2024-09-05T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.419+0000 7fd3d37fe640 1 -- 172.21.15.103:0/2299155687 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd3cc004ba0 con 0x7fd3dc101c50 2024-09-05T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.419+0000 7fd3e707d740 1 Processor -- start 2024-09-05T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.419+0000 7fd3e707d740 1 -- start start 2024-09-05T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.419+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e709a0 0x565352e70da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.419+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e712e0 0x565352e75da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.419+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e76370 0x565352e78830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.419+0000 7fd3e707d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565352dd42f0 con 0x565352e712e0 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.419+0000 7fd3e707d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565352d7c270 con 0x565352e76370 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.419+0000 7fd3e707d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565352dd02b0 con 0x565352e709a0 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2082640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e712e0 0x565352e75da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2082640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e712e0 0x565352e75da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32786/0 (socket says 172.21.15.103:32786) 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2082640 1 -- 172.21.15.103:0/1089372401 learned_addr learned my addr 172.21.15.103:0/1089372401 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e1881640 1 --2- 172.21.15.103:0/1089372401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e76370 0x565352e78830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2883640 1 --2- 172.21.15.103:0/1089372401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e709a0 0x565352e70da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3d37fe640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3c80a2c30 0x7fd3c80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2082640 1 -- 172.21.15.103:0/1089372401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e709a0 msgr2=0x565352e70da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2082640 1 --2- 172.21.15.103:0/1089372401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e709a0 0x565352e70da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2082640 1 -- 172.21.15.103:0/1089372401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e76370 msgr2=0x565352e78830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2082640 1 --2- 172.21.15.103:0/1089372401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e76370 0x565352e78830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2082640 1 -- 172.21.15.103:0/1089372401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565352d7b340 con 0x565352e712e0 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2883640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3c80a2c30 0x7fd3c80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3d37fe640 1 -- 172.21.15.103:0/2299155687 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd3cc0cf1a0 con 0x7fd3dc101c50 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3e2082640 1 --2- 172.21.15.103:0/1089372401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e712e0 0x565352e75da0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80314e0 tx=0x7fd3d8032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5023432935934e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3d0bfe640 1 -- 172.21.15.103:0/1089372401 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d803f070 con 0x565352e712e0 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3d0bfe640 1 -- 172.21.15.103:0/1089372401 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d8032e50 con 0x565352e712e0 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.420+0000 7fd3d0bfe640 1 -- 172.21.15.103:0/1089372401 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8035070 con 0x565352e712e0 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 -- 172.21.15.103:0/1089372401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e712e0 msgr2=0x565352e75da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e2883640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3c80a2c30 0x7fd3c80a50f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009e80 tx=0x7fd3d4004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 --2- 172.21.15.103:0/1089372401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e712e0 0x565352e75da0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80314e0 tx=0x7fd3d8032c60 comp rx=0 tx=0).stop 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 -- 172.21.15.103:0/1089372401 shutdown_connections 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 --2- 172.21.15.103:0/1089372401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e76370 0x565352e78830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 --2- 172.21.15.103:0/1089372401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e712e0 0x565352e75da0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 --2- 172.21.15.103:0/1089372401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e709a0 0x565352e70da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 -- 172.21.15.103:0/1089372401 >> 172.21.15.103:0/1089372401 conn(0x565352e6c010 msgr2=0x565352e6c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 -- 172.21.15.103:0/1089372401 shutdown_connections 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 -- 172.21.15.103:0/1089372401 wait complete. 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.421+0000 7fd3e707d740 1 -- start start 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e707d740 1 Processor -- start 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e709a0 0x565352e7ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e712e0 0x565352e7b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e707d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565352e792e0 con 0x565352e76370 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e707d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565352d5d290 con 0x565352e709a0 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e707d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565352e79460 con 0x565352e712e0 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e707d740 10 client.0 _ll_register_callbacks cb 0x565352d8e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2883640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2883640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e820b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32798/0 (socket says 172.21.15.103:32798) 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2883640 1 -- 172.21.15.103:0/3155743177 learned_addr learned my addr 172.21.15.103:0/3155743177 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e1881640 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e712e0 0x565352e7b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2082640 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e709a0 0x565352e7ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2883640 1 -- 172.21.15.103:0/3155743177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e712e0 msgr2=0x565352e7b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2883640 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e712e0 0x565352e7b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2883640 1 -- 172.21.15.103:0/3155743177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e709a0 msgr2=0x565352e7ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2883640 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e709a0 0x565352e7ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2883640 1 -- 172.21.15.103:0/3155743177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565352ae7210 con 0x565352e76370 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e1881640 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e712e0 0x565352e7b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3e2883640 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e820b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4017a50 tx=0x7fd3d4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849b8166682ced0c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.422+0000 7fd3c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/3155743177 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4025060 con 0x565352e76370 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/3155743177 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd3d4012050 con 0x565352e76370 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e707d740 1 -- 172.21.15.103:0/3155743177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d8032e50 con 0x565352e76370 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/3155743177 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3d402c050 con 0x565352e76370 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e707d740 10 client.15381 fetch_fsmap learned FSMap version 29 2024-09-05T02:02:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e707d740 1 -- 172.21.15.103:0/3155743177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565352bb3620 con 0x565352e76370 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/3155743177 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d8032e50 con 0x565352e76370 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/3155743177 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd3d4014070 con 0x565352e76370 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e707d740 10 client.15381 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e707d740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e707d740 20 client.15381 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e707d740 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565352e867a0 0x565352e88b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e707d740 4 client.15381 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e707d740 1 -- 172.21.15.103:0/3155743177 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565352de1a20 con 0x565352e867a0 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.423+0000 7fd3e2082640 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565352e867a0 0x565352e88b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.424+0000 7fd3c5ffb640 10 client.15381 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.424+0000 7fd3e2082640 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565352e867a0 0x565352e88b90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2e5a4a119aaa0f server_cookie=688e5e06934634df in_seq=0 out_seq=0 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/3155743177 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565352de1a20 con 0x565352e867a0 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3c5ffb640 10 client.15381 handle_command_reply: tid=0 2024-09-05T02:02:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 client.15381 shutdown 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 -- 172.21.15.103:0/3155743177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 msgr2=0x565352e820b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e820b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4017a50 tx=0x7fd3d4009540 comp rx=0 tx=0).stop 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 -- 172.21.15.103:0/3155743177 shutdown_connections 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565352e867a0 0x565352e88b90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e820b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e712e0 0x565352e7b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3c5ffb640 0 client.15381 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 --2- 172.21.15.103:0/3155743177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e709a0 0x565352e7ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 -- 172.21.15.103:0/3155743177 >> 172.21.15.103:0/3155743177 conn(0x565352e6c010 msgr2=0x565352e6d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 -- 172.21.15.103:0/3155743177 shutdown_connections 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 1 -- 172.21.15.103:0/3155743177 wait complete. 2024-09-05T02:02:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.425+0000 7fd3e707d740 20 client.15381 trim_cache size 0 max 16384 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e707d740 1 Processor -- start 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e707d740 1 -- start start 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e709a0 0x565352e6cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 0x565352e82070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e6d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e707d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1aca00 con 0x565352e76370 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e707d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ae540 con 0x565352e712e0 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e707d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ad0d0 con 0x565352e709a0 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e2883640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e6d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e2883640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e6d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32806/0 (socket says 172.21.15.103:32806) 2024-09-05T02:02:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e2883640 1 -- 172.21.15.103:0/1000318401 learned_addr learned my addr 172.21.15.103:0/1000318401 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e1881640 1 --2- 172.21.15.103:0/1000318401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 0x565352e82070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e2883640 1 -- 172.21.15.103:0/1000318401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e709a0 msgr2=0x565352e6cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e2883640 1 --2- 172.21.15.103:0/1000318401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e709a0 0x565352e6cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e2883640 1 -- 172.21.15.103:0/1000318401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 msgr2=0x565352e82070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e2883640 1 --2- 172.21.15.103:0/1000318401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 0x565352e82070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.439+0000 7fd3e2883640 1 -- 172.21.15.103:0/1000318401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565352e88f00 con 0x565352e76370 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e2883640 1 --2- 172.21.15.103:0/1000318401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e6d400 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4017470 tx=0x7fd3d4009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75330faa01c8459a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3c7fff640 1 -- 172.21.15.103:0/1000318401 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4025040 con 0x565352e76370 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3c7fff640 1 -- 172.21.15.103:0/1000318401 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d4020bb0 con 0x565352e76370 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3c7fff640 1 -- 172.21.15.103:0/1000318401 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400ad80 con 0x565352e76370 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e707d740 1 -- 172.21.15.103:0/1000318401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 msgr2=0x565352e6d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e707d740 1 --2- 172.21.15.103:0/1000318401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e6d400 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4017470 tx=0x7fd3d4009b90 comp rx=0 tx=0).stop 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e707d740 1 -- 172.21.15.103:0/1000318401 shutdown_connections 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e707d740 1 --2- 172.21.15.103:0/1000318401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e76370 0x565352e6d400 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e707d740 1 --2- 172.21.15.103:0/1000318401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 0x565352e82070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e707d740 1 --2- 172.21.15.103:0/1000318401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e709a0 0x565352e6cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e707d740 1 -- 172.21.15.103:0/1000318401 >> 172.21.15.103:0/1000318401 conn(0x565352e6c010 msgr2=0x565352e7a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e707d740 1 -- 172.21.15.103:0/1000318401 shutdown_connections 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.440+0000 7fd3e707d740 1 -- 172.21.15.103:0/1000318401 wait complete. 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e707d740 1 -- start start 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e707d740 1 Processor -- start 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e709a0 0x565352e7f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 0x565352e7fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e707d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e76370 0x565352e80cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e707d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ade70 con 0x565352e709a0 2024-09-05T02:02:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e707d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ad7a0 con 0x565352e712e0 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e707d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1aa9e0 con 0x565352e76370 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e707d740 10 client.0 _ll_register_callbacks cb 0x7fd3d4009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2082640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e709a0 0x565352e7f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2082640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e709a0 0x565352e7f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32822/0 (socket says 172.21.15.103:32822) 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2082640 1 -- 172.21.15.103:0/2376729553 learned_addr learned my addr 172.21.15.103:0/2376729553 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e1881640 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 0x565352e7fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2883640 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e76370 0x565352e80cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2082640 1 -- 172.21.15.103:0/2376729553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e76370 msgr2=0x565352e80cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2082640 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e76370 0x565352e80cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2082640 1 -- 172.21.15.103:0/2376729553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 msgr2=0x565352e7fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2082640 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 0x565352e7fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2082640 1 -- 172.21.15.103:0/2376729553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565352e81300 con 0x565352e709a0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e1881640 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 0x565352e7fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3e2082640 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e709a0 0x565352e7f7f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009230 tx=0x7fd3d80092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f9056ec5c5d7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.441+0000 7fd3c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/2376729553 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8035040 con 0x565352e709a0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/2376729553 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd3d8004440 con 0x565352e709a0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e707d740 1 -- 172.21.15.103:0/2376729553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d4020bb0 con 0x565352e709a0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/2376729553 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3d805b050 con 0x565352e709a0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e707d740 10 client.15387 fetch_fsmap learned FSMap version 29 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e707d740 1 -- 172.21.15.103:0/2376729553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565352ef1520 con 0x565352e709a0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/2376729553 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d4020bb0 con 0x565352e709a0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/2376729553 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd3d8003990 con 0x565352e709a0 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e707d740 10 client.15387 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:02:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e707d740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:02:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e707d740 20 client.15387 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e707d740 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565352ef1a90 0x565352ef3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e707d740 4 client.15387 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:02:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e707d740 1 -- 172.21.15.103:0/2376729553 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565352ddbd70 con 0x565352ef1a90 2024-09-05T02:02:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.442+0000 7fd3e1881640 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565352ef1a90 0x565352ef3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.443+0000 7fd3e1881640 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565352ef1a90 0x565352ef3e80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47063cfc3903e44b server_cookie=ef4b90dbd6fc3b30 in_seq=0 out_seq=0 2024-09-05T02:02:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.443+0000 7fd3c5ffb640 10 client.15387 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.456+0000 7fd3c5ffb640 1 -- 172.21.15.103:0/2376729553 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565352ddbd70 con 0x565352ef1a90 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3c5ffb640 10 client.15387 handle_command_reply: tid=0 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 client.15387 shutdown 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 -- 172.21.15.103:0/2376729553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e709a0 msgr2=0x565352e7f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e709a0 0x565352e7f7f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009230 tx=0x7fd3d80092d0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 -- 172.21.15.103:0/2376729553 shutdown_connections 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565352ef1a90 0x565352ef3e80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565352e76370 0x565352e80cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565352e712e0 0x565352e7fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3c5ffb640 0 client.15387 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 --2- 172.21.15.103:0/2376729553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565352e709a0 0x565352e7f7f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 -- 172.21.15.103:0/2376729553 >> 172.21.15.103:0/2376729553 conn(0x565352e6c010 msgr2=0x565352e6a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 -- 172.21.15.103:0/2376729553 shutdown_connections 2024-09-05T02:02:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 1 -- 172.21.15.103:0/2376729553 wait complete. 2024-09-05T02:02:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e707d740 20 client.15387 trim_cache size 0 max 16384 2024-09-05T02:02:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:02:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:02:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:02:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:02:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3c80a2c30 msgr2=0x7fd3c80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3c80a2c30 0x7fd3c80a50f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009e80 tx=0x7fd3d4004b50 comp rx=0 tx=0).stop 2024-09-05T02:02:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc101c50 msgr2=0x7fd3dc103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.457+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc101c50 0x7fd3dc103030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00c890 tx=0x7fd3cc00cd50 comp rx=0 tx=0).stop 2024-09-05T02:02:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.458+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2299155687 shutdown_connections 2024-09-05T02:02:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.458+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3c80a2c30 0x7fd3c80a50f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.458+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3dc102610 0x7fd3dc103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.458+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3dc101c50 0x7fd3dc103030 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.458+0000 7fd3e42e6640 1 --2- 172.21.15.103:0/2299155687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3dc100a60 0x7fd3dc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.458+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2299155687 >> 172.21.15.103:0/2299155687 conn(0x7fd3dc0fc1e0 msgr2=0x7fd3dc06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.458+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2299155687 shutdown_connections 2024-09-05T02:02:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:02:54.458+0000 7fd3e42e6640 1 -- 172.21.15.103:0/2299155687 wait complete. 2024-09-05T02:02:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:54 smithi167 ceph-mon[35122]: pgmap v364: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-05T02:02:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:54 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2243934584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:54 smithi139 ceph-mon[34262]: pgmap v364: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-05T02:02:54.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:54 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2243934584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:54 smithi103 ceph-mon[30902]: pgmap v364: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-05T02:02:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:54 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2243934584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:02:54.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T02:00:15.086875+0000', 'modified': '2024-09-05T02:02:41.148036+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24497, 'mds_1': 24466, 'mds_2': 24464, 'mds_3': 24457, 'mds_4': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/1603518036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 1603518036}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 1603518036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/426074448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 426074448}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 426074448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6833/4042514104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 4042514104}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 4042514104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/2095779466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 2095779466}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 2095779466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/1110459707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 1110459707}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 1110459707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.139:6835/3204357322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 3204357322}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 3204357322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.167:6837/3468047781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 3468047781}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3468047781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.139:6837/2336570561', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 2336570561}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 2336570561}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.167:6839/1858062899', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 1858062899}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 1858062899}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.139:6839/3082363979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 3082363979}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 3082363979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24449, 24457, 24464, 24466, 24497]} 2024-09-05T02:02:54.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-05T02:02:54.700 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.e status 2024-09-05T02:02:54.985 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:55.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.506+0000 7fbada57c640 1 -- 172.21.15.103:0/1217932462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4106a80 msgr2=0x7fbad4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:55.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.506+0000 7fbada57c640 1 --2- 172.21.15.103:0/1217932462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4106a80 0x7fbad4106e60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009930 tx=0x7fbac402a420 comp rx=0 tx=0).stop 2024-09-05T02:02:55.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.506+0000 7fbada57c640 1 -- 172.21.15.103:0/1217932462 shutdown_connections 2024-09-05T02:02:55.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.506+0000 7fbada57c640 1 --2- 172.21.15.103:0/1217932462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad4101420 0x7fbad410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.506+0000 7fbada57c640 1 --2- 172.21.15.103:0/1217932462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad4100a80 0x7fbad4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.506+0000 7fbada57c640 1 --2- 172.21.15.103:0/1217932462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4106a80 0x7fbad4106e60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.506+0000 7fbada57c640 1 -- 172.21.15.103:0/1217932462 >> 172.21.15.103:0/1217932462 conn(0x7fbad40fc7a0 msgr2=0x7fbad40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:55.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.506+0000 7fbada57c640 1 -- 172.21.15.103:0/1217932462 shutdown_connections 2024-09-05T02:02:55.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.506+0000 7fbada57c640 1 -- 172.21.15.103:0/1217932462 wait complete. 2024-09-05T02:02:55.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbada57c640 1 Processor -- start 2024-09-05T02:02:55.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbada57c640 1 -- start start 2024-09-05T02:02:55.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbada57c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad4100a80 0x7fbad4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbada57c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4101420 0x7fbad41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbada57c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad4106a80 0x7fbad419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbada57c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4110d70 con 0x7fbad4101420 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbada57c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4110bf0 con 0x7fbad4106a80 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbada57c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4110ef0 con 0x7fbad4100a80 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4101420 0x7fbad41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4101420 0x7fbad41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32838/0 (socket says 172.21.15.103:32838) 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad37fe640 1 -- 172.21.15.103:0/3876738076 learned_addr learned my addr 172.21.15.103:0/3876738076 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad3fff640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad4100a80 0x7fbad4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad8b19640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad4106a80 0x7fbad419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad37fe640 1 -- 172.21.15.103:0/3876738076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad4100a80 msgr2=0x7fbad4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad37fe640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad4100a80 0x7fbad4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad37fe640 1 -- 172.21.15.103:0/3876738076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad4106a80 msgr2=0x7fbad419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad37fe640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad4106a80 0x7fbad419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.507+0000 7fbad37fe640 1 -- 172.21.15.103:0/3876738076 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad419d410 con 0x7fbad4101420 2024-09-05T02:02:55.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.508+0000 7fbad37fe640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4101420 0x7fbad41989a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00c970 tx=0x7fbabc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:55.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.508+0000 7fbad17fa640 1 -- 172.21.15.103:0/3876738076 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc007bb0 con 0x7fbad4101420 2024-09-05T02:02:55.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.508+0000 7fbada57c640 1 -- 172.21.15.103:0/3876738076 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad419d700 con 0x7fbad4101420 2024-09-05T02:02:55.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.508+0000 7fbad17fa640 1 -- 172.21.15.103:0/3876738076 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbabc007d50 con 0x7fbad4101420 2024-09-05T02:02:55.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.508+0000 7fbad17fa640 1 -- 172.21.15.103:0/3876738076 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00f450 con 0x7fbad4101420 2024-09-05T02:02:55.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.508+0000 7fbada57c640 1 -- 172.21.15.103:0/3876738076 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41a4fe0 con 0x7fbad4101420 2024-09-05T02:02:55.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.510+0000 7fbad17fa640 1 -- 172.21.15.103:0/3876738076 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbabc004f20 con 0x7fbad4101420 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.510+0000 7fbadc955740 1 Processor -- start 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbadc955740 1 -- start start 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbadc955740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fc9d0 0x563fa35fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbadc955740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa3607670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbadc955740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 0x563fa3609fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbadc955740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563fa3563090 con 0x563fa35fd2f0 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbadc955740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563fa350aa70 con 0x563fa35fc9d0 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbadc955740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563fa355f050 con 0x563fa3607bb0 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad17fa640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbac00a2b80 0x7fbac00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad8b19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fc9d0 0x563fa35fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad8b19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fc9d0 0x563fa35fcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:43156/0 (socket says 172.21.15.103:43156) 2024-09-05T02:02:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad8b19640 1 -- 172.21.15.103:0/865567261 learned_addr learned my addr 172.21.15.103:0/865567261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad8b19640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbac00a2b80 0x7fbac00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad3fff640 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa3607670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad17fa640 1 -- 172.21.15.103:0/3876738076 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbabc0cf170 con 0x7fbad4101420 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad37fe640 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 0x563fa3609fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad8b19640 1 -- 172.21.15.103:0/865567261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 msgr2=0x563fa3609fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad8b19640 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 0x563fa3609fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad8b19640 1 -- 172.21.15.103:0/865567261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 msgr2=0x563fa3607670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad8b19640 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa3607670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad8b19640 1 -- 172.21.15.103:0/865567261 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa3517130 con 0x563fa35fc9d0 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad37fe640 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 0x563fa3609fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.511+0000 7fbad3fff640 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa3607670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.512+0000 7fbad8b19640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbac00a2b80 0x7fbac00a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009fd0 tx=0x7fbacc01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:55.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.512+0000 7fbad8b19640 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fc9d0 0x563fa35fcdb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00af30 tx=0x7fbacc016700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76996149c520bf4a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.512+0000 7fbaca7fc640 1 -- 172.21.15.103:0/865567261 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc026070 con 0x563fa35fc9d0 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.512+0000 7fbaca7fc640 1 -- 172.21.15.103:0/865567261 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbacc00a590 con 0x563fa35fc9d0 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.512+0000 7fbaca7fc640 1 -- 172.21.15.103:0/865567261 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc013040 con 0x563fa35fc9d0 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 -- 172.21.15.103:0/865567261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fc9d0 msgr2=0x563fa35fcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fc9d0 0x563fa35fcdb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00af30 tx=0x7fbacc016700 comp rx=0 tx=0).stop 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 -- 172.21.15.103:0/865567261 shutdown_connections 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 0x563fa3609fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa3607670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 --2- 172.21.15.103:0/865567261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fc9d0 0x563fa35fcdb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 -- 172.21.15.103:0/865567261 >> 172.21.15.103:0/865567261 conn(0x563fa35f8f60 msgr2=0x563fa35f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 -- 172.21.15.103:0/865567261 shutdown_connections 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 -- 172.21.15.103:0/865567261 wait complete. 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 -- start start 2024-09-05T02:02:55.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.513+0000 7fbadc955740 1 Processor -- start 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbadc955740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 0x563fa36074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbadc955740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa36047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbadc955740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 0x563fa3604d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbadc955740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563fa360a780 con 0x563fa35fd2f0 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbadc955740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563fa351df20 con 0x563fa3607bb0 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbadc955740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563fa360a900 con 0x563fa35fc9d0 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbadc955740 10 client.0 _ll_register_callbacks cb 0x563fa34e7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa36047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa36047f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32862/0 (socket says 172.21.15.103:32862) 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad37fe640 1 -- 172.21.15.103:0/2902003775 learned_addr learned my addr 172.21.15.103:0/2902003775 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad8b19640 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 0x563fa3604d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad3fff640 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 0x563fa36074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad37fe640 1 -- 172.21.15.103:0/2902003775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 msgr2=0x563fa36074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad37fe640 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 0x563fa36074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad37fe640 1 -- 172.21.15.103:0/2902003775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 msgr2=0x563fa3604d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad37fe640 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 0x563fa3604d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad37fe640 1 -- 172.21.15.103:0/2902003775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa32d42b0 con 0x563fa35fd2f0 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad3fff640 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 0x563fa36074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbad37fe640 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa36047f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbabc09b5b0 tx=0x7fbabc096050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f179eaf733628 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbab37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbab37fe640 1 -- 172.21.15.103:0/2902003775 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0b9060 con 0x563fa35fd2f0 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbab37fe640 1 -- 172.21.15.103:0/2902003775 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbabc0a3e00 con 0x563fa35fd2f0 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.514+0000 7fbadc955740 1 -- 172.21.15.103:0/2902003775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbacc00a590 con 0x563fa35fd2f0 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbab37fe640 1 -- 172.21.15.103:0/2902003775 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbabc081050 con 0x563fa35fd2f0 2024-09-05T02:02:55.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbadc955740 10 client.15399 fetch_fsmap learned FSMap version 29 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbadc955740 1 -- 172.21.15.103:0/2902003775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fa36160c0 con 0x563fa35fd2f0 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbab37fe640 1 -- 172.21.15.103:0/2902003775 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbacc00a590 con 0x563fa35fd2f0 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbab37fe640 1 -- 172.21.15.103:0/2902003775 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbabc084bb0 con 0x563fa35fd2f0 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbadc955740 10 client.15399 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbadc955740 10 client.15399 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbadc955740 20 client.15399 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbadc955740 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x563fa3618620 0x563fa3619a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbadc955740 4 client.15399 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbadc955740 1 -- 172.21.15.103:0/2902003775 --> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fa3570600 con 0x563fa3618620 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.515+0000 7fbad3fff640 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x563fa3618620 0x563fa3619a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.516+0000 7fbab37fe640 10 client.15399 ms_handle_connect on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.516+0000 7fbad3fff640 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x563fa3618620 0x563fa3619a90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e31ab2b04f2b71f6 server_cookie=6bf76cb6d228c8e4 in_seq=0 out_seq=0 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.516+0000 7fbab37fe640 1 -- 172.21.15.103:0/2902003775 <== mds.4 v2:172.21.15.139:6834/3204357322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fa3570600 con 0x563fa3618620 2024-09-05T02:02:55.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.516+0000 7fbab37fe640 10 client.15399 handle_command_reply: tid=0 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.516+0000 7fbadc955740 1 client.15399 shutdown 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 -- 172.21.15.103:0/2902003775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 msgr2=0x563fa36047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa36047f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbabc09b5b0 tx=0x7fbabc096050 comp rx=0 tx=0).stop 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 -- 172.21.15.103:0/2902003775 shutdown_connections 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x563fa3618620 0x563fa3619a90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 0x563fa3604d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa36047f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 --2- 172.21.15.103:0/2902003775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 0x563fa36074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbab37fe640 0 client.15399 ms_handle_reset on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 -- 172.21.15.103:0/2902003775 >> 172.21.15.103:0/2902003775 conn(0x563fa35f8f60 msgr2=0x563fa35fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 -- 172.21.15.103:0/2902003775 shutdown_connections 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 1 -- 172.21.15.103:0/2902003775 wait complete. 2024-09-05T02:02:55.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.517+0000 7fbadc955740 20 client.15399 trim_cache size 0 max 16384 2024-09-05T02:02:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbadc955740 1 Processor -- start 2024-09-05T02:02:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbadc955740 1 -- start start 2024-09-05T02:02:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbadc955740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 0x563fa36193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbadc955740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa35f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbadc955740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 0x563fa36030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbadc955740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563fa3617dd0 con 0x563fa35fd2f0 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbadc955740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563fa36184a0 con 0x563fa3607bb0 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbadc955740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563fa3618b70 con 0x563fa35fc9d0 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa35f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa35f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32864/0 (socket says 172.21.15.103:32864) 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbad37fe640 1 -- 172.21.15.103:0/2523138131 learned_addr learned my addr 172.21.15.103:0/2523138131 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbad8b19640 1 --2- 172.21.15.103:0/2523138131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 0x563fa36030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbad37fe640 1 -- 172.21.15.103:0/2523138131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 msgr2=0x563fa36193c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.534+0000 7fbad37fe640 1 --2- 172.21.15.103:0/2523138131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 0x563fa36193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbad37fe640 1 -- 172.21.15.103:0/2523138131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 msgr2=0x563fa36030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbad37fe640 1 --2- 172.21.15.103:0/2523138131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 0x563fa36030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbad37fe640 1 -- 172.21.15.103:0/2523138131 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa360a890 con 0x563fa35fd2f0 2024-09-05T02:02:55.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbad8b19640 1 --2- 172.21.15.103:0/2523138131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 0x563fa36030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:55.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbad37fe640 1 --2- 172.21.15.103:0/2523138131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa35f3ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0ccc30 tx=0x7fbabc086180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bd6cb925666813 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:55.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbac9ffb640 1 -- 172.21.15.103:0/2523138131 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc01d070 con 0x563fa35fd2f0 2024-09-05T02:02:55.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbac9ffb640 1 -- 172.21.15.103:0/2523138131 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbabc09bd20 con 0x563fa35fd2f0 2024-09-05T02:02:55.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbac9ffb640 1 -- 172.21.15.103:0/2523138131 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc09f9d0 con 0x563fa35fd2f0 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbadc955740 1 -- 172.21.15.103:0/2523138131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 msgr2=0x563fa35f3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbadc955740 1 --2- 172.21.15.103:0/2523138131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa35f3ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0ccc30 tx=0x7fbabc086180 comp rx=0 tx=0).stop 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbadc955740 1 -- 172.21.15.103:0/2523138131 shutdown_connections 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbadc955740 1 --2- 172.21.15.103:0/2523138131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa3607bb0 0x563fa36030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbadc955740 1 --2- 172.21.15.103:0/2523138131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fd2f0 0x563fa35f3ce0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbadc955740 1 --2- 172.21.15.103:0/2523138131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa35fc9d0 0x563fa36193c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.535+0000 7fbadc955740 1 -- 172.21.15.103:0/2523138131 >> 172.21.15.103:0/2523138131 conn(0x563fa35f8f60 msgr2=0x563fa35f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.536+0000 7fbadc955740 1 -- 172.21.15.103:0/2523138131 shutdown_connections 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.536+0000 7fbadc955740 1 -- 172.21.15.103:0/2523138131 wait complete. 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.536+0000 7fbadc955740 1 -- start start 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.536+0000 7fbadc955740 1 Processor -- start 2024-09-05T02:02:55.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.536+0000 7fbadc955740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fc9d0 0x563fa3622340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.536+0000 7fbadc955740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fd2f0 0x563fa361b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.536+0000 7fbadc955740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 0x563fa361bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.536+0000 7fbadc955740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563fa3619910 con 0x563fa35fc9d0 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.536+0000 7fbadc955740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563fa3619240 con 0x563fa35fd2f0 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbadc955740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563fa3619fe0 con 0x563fa3607bb0 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fc9d0 0x563fa3622340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbadc955740 10 client.0 _ll_register_callbacks cb 0x7fbabc0058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fc9d0 0x563fa3622340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32874/0 (socket says 172.21.15.103:32874) 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad3fff640 1 -- 172.21.15.103:0/999805037 learned_addr learned my addr 172.21.15.103:0/999805037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad37fe640 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fd2f0 0x563fa361b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad8b19640 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 0x563fa361bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad3fff640 1 -- 172.21.15.103:0/999805037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 msgr2=0x563fa361bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad3fff640 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 0x563fa361bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad3fff640 1 -- 172.21.15.103:0/999805037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fd2f0 msgr2=0x563fa361b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad3fff640 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fd2f0 0x563fa361b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad3fff640 1 -- 172.21.15.103:0/999805037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa361c220 con 0x563fa35fc9d0 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbad3fff640 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fc9d0 0x563fa3622340 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbac402a460 tx=0x7fbac4032cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65052d4dd9af70c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbab37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbab37fe640 1 -- 172.21.15.103:0/999805037 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4035040 con 0x563fa35fc9d0 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbab37fe640 1 -- 172.21.15.103:0/999805037 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbac40035f0 con 0x563fa35fc9d0 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbadc955740 1 -- 172.21.15.103:0/999805037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbabc09bd20 con 0x563fa35fc9d0 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbab37fe640 1 -- 172.21.15.103:0/999805037 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbac4056050 con 0x563fa35fc9d0 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbadc955740 10 client.15405 fetch_fsmap learned FSMap version 29 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.537+0000 7fbadc955740 1 -- 172.21.15.103:0/999805037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fa3682d30 con 0x563fa35fc9d0 2024-09-05T02:02:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbab37fe640 1 -- 172.21.15.103:0/999805037 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbabc09bd20 con 0x563fa35fc9d0 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbab37fe640 1 -- 172.21.15.103:0/999805037 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbac4032540 con 0x563fa35fc9d0 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbadc955740 10 client.15405 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbadc955740 10 client.15405 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbadc955740 20 client.15405 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbadc955740 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x563fa3687750 0x563fa3686460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbadc955740 4 client.15405 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbadc955740 1 -- 172.21.15.103:0/999805037 --> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] -- command(tid 0: {"prefix": "status"}) -- 0x563fa356ab10 con 0x563fa3687750 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbad37fe640 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x563fa3687750 0x563fa3686460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbad37fe640 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x563fa3687750 0x563fa3686460 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e930ccf2e0ff5893 server_cookie=c164b1f0ee2cd7e1 in_seq=0 out_seq=0 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.538+0000 7fbab37fe640 10 client.15405 ms_handle_connect on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbab37fe640 1 -- 172.21.15.103:0/999805037 <== mds.4 v2:172.21.15.139:6834/3204357322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x563fa356ab10 con 0x563fa3687750 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbab37fe640 10 client.15405 handle_command_reply: tid=0 2024-09-05T02:02:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 client.15405 shutdown 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 -- 172.21.15.103:0/999805037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fc9d0 msgr2=0x563fa3622340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fc9d0 0x563fa3622340 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbac402a460 tx=0x7fbac4032cf0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 -- 172.21.15.103:0/999805037 shutdown_connections 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fa3607bb0 0x563fa361bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.139:6834/3204357322,v1:172.21.15.139:6835/3204357322] conn(0x563fa3687750 0x563fa3686460 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fa35fd2f0 0x563fa361b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 --2- 172.21.15.103:0/999805037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa35fc9d0 0x563fa3622340 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbab37fe640 0 client.15405 ms_handle_reset on v2:172.21.15.139:6834/3204357322 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 -- 172.21.15.103:0/999805037 >> 172.21.15.103:0/999805037 conn(0x563fa35f8f60 msgr2=0x563fa35fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 -- 172.21.15.103:0/999805037 shutdown_connections 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 1 -- 172.21.15.103:0/999805037 wait complete. 2024-09-05T02:02:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.539+0000 7fbadc955740 20 client.15405 trim_cache size 0 max 16384 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24449, 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 117.319278123, 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 29, 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T02:02:55.547 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 193.050097561 2024-09-05T02:02:55.548 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 -- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbac00a2b80 msgr2=0x7fbac00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbac00a2b80 0x7fbac00a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009fd0 tx=0x7fbacc01a000 comp rx=0 tx=0).stop 2024-09-05T02:02:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 -- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4101420 msgr2=0x7fbad41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4101420 0x7fbad41989a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00c970 tx=0x7fbabc00ce30 comp rx=0 tx=0).stop 2024-09-05T02:02:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 -- 172.21.15.103:0/3876738076 shutdown_connections 2024-09-05T02:02:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad4106a80 0x7fbad419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbad4101420 0x7fbad41989a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbac00a2b80 0x7fbac00a5040 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 --2- 172.21.15.103:0/3876738076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad4100a80 0x7fbad4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 -- 172.21.15.103:0/3876738076 >> 172.21.15.103:0/3876738076 conn(0x7fbad40fc7a0 msgr2=0x7fbad406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 -- 172.21.15.103:0/3876738076 shutdown_connections 2024-09-05T02:02:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:55.540+0000 7fbada57c640 1 -- 172.21.15.103:0/3876738076 wait complete. 2024-09-05T02:02:56.279 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.c status 2024-09-05T02:02:56.528 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:56 smithi167 ceph-mon[35122]: pgmap v365: 129 pgs: 129 active+clean; 657 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-09-05T02:02:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:56 smithi139 ceph-mon[34262]: pgmap v365: 129 pgs: 129 active+clean; 657 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-09-05T02:02:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:56 smithi103 ceph-mon[30902]: pgmap v365: 129 pgs: 129 active+clean; 657 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-09-05T02:02:57.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 -- 172.21.15.103:0/2797633141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c102ef0 msgr2=0x7fa43c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:57.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 --2- 172.21.15.103:0/2797633141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c102ef0 0x7fa43c10da40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009ca0 tx=0x7fa42c02a950 comp rx=0 tx=0).stop 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 -- 172.21.15.103:0/2797633141 shutdown_connections 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 --2- 172.21.15.103:0/2797633141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c102ef0 0x7fa43c10da40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 --2- 172.21.15.103:0/2797633141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c102570 0x7fa43c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 --2- 172.21.15.103:0/2797633141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa43c100e20 0x7fa43c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 -- 172.21.15.103:0/2797633141 >> 172.21.15.103:0/2797633141 conn(0x7fa43c0fc7d0 msgr2=0x7fa43c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 -- 172.21.15.103:0/2797633141 shutdown_connections 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 -- 172.21.15.103:0/2797633141 wait complete. 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 Processor -- start 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.987+0000 7fa443656640 1 -- start start 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa443656640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c100e20 0x7fa43c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa443656640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa43c102570 0x7fa43c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa443656640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c102ef0 0x7fa43c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa443656640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c110e70 con 0x7fa43c100e20 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa443656640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c110cf0 con 0x7fa43c102ef0 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa443656640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c110ff0 con 0x7fa43c102570 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa4413f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c100e20 0x7fa43c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa4413f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c100e20 0x7fa43c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32894/0 (socket says 172.21.15.103:32894) 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa4413f2640 1 -- 172.21.15.103:0/1191269288 learned_addr learned my addr 172.21.15.103:0/1191269288 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa440bf1640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa43c102570 0x7fa43c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa441bf3640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c102ef0 0x7fa43c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa4413f2640 1 -- 172.21.15.103:0/1191269288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa43c102570 msgr2=0x7fa43c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa4413f2640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa43c102570 0x7fa43c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa4413f2640 1 -- 172.21.15.103:0/1191269288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c102ef0 msgr2=0x7fa43c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa4413f2640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c102ef0 0x7fa43c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa4413f2640 1 -- 172.21.15.103:0/1191269288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c1a6a10 con 0x7fa43c100e20 2024-09-05T02:02:57.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa440bf1640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa43c102570 0x7fa43c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:57.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa441bf3640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c102ef0 0x7fa43c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:57.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa4413f2640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c100e20 0x7fa43c0fcc80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa43400d770 tx=0x7fa43400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:57.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa43a7fc640 1 -- 172.21.15.103:0/1191269288 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434017070 con 0x7fa43c100e20 2024-09-05T02:02:57.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa443656640 1 -- 172.21.15.103:0/1191269288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c1a6d00 con 0x7fa43c100e20 2024-09-05T02:02:57.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa43a7fc640 1 -- 172.21.15.103:0/1191269288 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4340054d0 con 0x7fa43c100e20 2024-09-05T02:02:57.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa43a7fc640 1 -- 172.21.15.103:0/1191269288 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434004cf0 con 0x7fa43c100e20 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.988+0000 7fa443656640 1 -- 172.21.15.103:0/1191269288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c1a71e0 con 0x7fa43c100e20 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa43a7fc640 1 -- 172.21.15.103:0/1191269288 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa434004e90 con 0x7fa43c100e20 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa445a2f740 1 Processor -- start 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa445a2f740 1 -- start start 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f28cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28d2f0 0x56209f297670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 0x56209f299fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa445a2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56209f1f3090 con 0x56209f28d2f0 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa445a2f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56209f19aa70 con 0x56209f28c9d0 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa445a2f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56209f1ef050 con 0x56209f297bb0 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa4413f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28d2f0 0x56209f297670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa4413f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28d2f0 0x56209f297670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32896/0 (socket says 172.21.15.103:32896) 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa4413f2640 1 -- 172.21.15.103:0/1448224371 learned_addr learned my addr 172.21.15.103:0/1448224371 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa43a7fc640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4240a2b80 0x7fa4240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.991+0000 7fa440bf1640 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 0x56209f299fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa441bf3640 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f28cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa441bf3640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4240a2b80 0x7fa4240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa43a7fc640 1 -- 172.21.15.103:0/1191269288 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa4340cef10 con 0x7fa43c100e20 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa4413f2640 1 -- 172.21.15.103:0/1448224371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 msgr2=0x56209f299fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa4413f2640 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 0x56209f299fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa4413f2640 1 -- 172.21.15.103:0/1448224371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 msgr2=0x56209f28cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa4413f2640 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f28cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa4413f2640 1 -- 172.21.15.103:0/1448224371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209f1a7130 con 0x56209f28d2f0 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa441bf3640 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f28cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:57.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa4413f2640 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28d2f0 0x56209f297670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa4340a3b00 tx=0x7fa434005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37323ce929b1d3ea server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa441bf3640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4240a2b80 0x7fa4240a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009a60 tx=0x7fa42c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa4237fe640 1 -- 172.21.15.103:0/1448224371 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434020030 con 0x56209f28d2f0 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa4237fe640 1 -- 172.21.15.103:0/1448224371 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa434009c30 con 0x56209f28d2f0 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.992+0000 7fa4237fe640 1 -- 172.21.15.103:0/1448224371 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43409b3c0 con 0x56209f28d2f0 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 -- 172.21.15.103:0/1448224371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28d2f0 msgr2=0x56209f297670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28d2f0 0x56209f297670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa4340a3b00 tx=0x7fa434005680 comp rx=0 tx=0).stop 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 -- 172.21.15.103:0/1448224371 shutdown_connections 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 0x56209f299fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28d2f0 0x56209f297670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 --2- 172.21.15.103:0/1448224371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f28cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 -- 172.21.15.103:0/1448224371 >> 172.21.15.103:0/1448224371 conn(0x56209f288f60 msgr2=0x56209f289340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 -- 172.21.15.103:0/1448224371 shutdown_connections 2024-09-05T02:02:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 -- 172.21.15.103:0/1448224371 wait complete. 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 -- start start 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.993+0000 7fa445a2f740 1 Processor -- start 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f297450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 0x56209f2947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f294ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa445a2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56209f29a780 con 0x56209f297bb0 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa445a2f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56209f1adf20 con 0x56209f28c9d0 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa445a2f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56209f29a900 con 0x56209f28d2f0 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa445a2f740 10 client.0 _ll_register_callbacks cb 0x56209f177030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa441bf3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f294ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa441bf3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f294ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32906/0 (socket says 172.21.15.103:32906) 2024-09-05T02:02:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa441bf3640 1 -- 172.21.15.103:0/783127921 learned_addr learned my addr 172.21.15.103:0/783127921 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa440bf1640 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 0x56209f2947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa4413f2640 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f297450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa441bf3640 1 -- 172.21.15.103:0/783127921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 msgr2=0x56209f2947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa441bf3640 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 0x56209f2947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa441bf3640 1 -- 172.21.15.103:0/783127921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 msgr2=0x56209f297450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa441bf3640 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f297450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa441bf3640 1 -- 172.21.15.103:0/783127921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209ef642b0 con 0x56209f297bb0 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa4413f2640 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f297450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa441bf3640 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f294ce0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa42c02f840 tx=0x7fa42c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ef0002ac1aa7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.994+0000 7fa420ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa420ff9640 1 -- 172.21.15.103:0/783127921 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c04c070 con 0x56209f297bb0 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa420ff9640 1 -- 172.21.15.103:0/783127921 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa42c004450 con 0x56209f297bb0 2024-09-05T02:02:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa445a2f740 1 -- 172.21.15.103:0/783127921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56209f200600 con 0x56209f297bb0 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa420ff9640 1 -- 172.21.15.103:0/783127921 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa42c05a050 con 0x56209f297bb0 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa445a2f740 10 client.15417 fetch_fsmap learned FSMap version 29 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa445a2f740 1 -- 172.21.15.103:0/783127921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56209f2a60d0 con 0x56209f297bb0 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa420ff9640 1 -- 172.21.15.103:0/783127921 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56209f200600 con 0x56209f297bb0 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa420ff9640 1 -- 172.21.15.103:0/783127921 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa42c030440 con 0x56209f297bb0 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa445a2f740 10 client.15417 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa445a2f740 10 client.15417 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa445a2f740 20 client.15417 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa445a2f740 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x56209f2a87a0 0x56209f2a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa445a2f740 4 client.15417 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.995+0000 7fa445a2f740 1 -- 172.21.15.103:0/783127921 --> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56209f1fab10 con 0x56209f2a87a0 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.996+0000 7fa4413f2640 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x56209f2a87a0 0x56209f2a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.996+0000 7fa4413f2640 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x56209f2a87a0 0x56209f2a9c10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=225ab9455b08837b server_cookie=4acca3a3d6e58cbb in_seq=0 out_seq=0 2024-09-05T02:02:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.996+0000 7fa420ff9640 10 client.15417 ms_handle_connect on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa420ff9640 1 -- 172.21.15.103:0/783127921 <== mds.3 v2:172.21.15.167:6836/3468047781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56209f1fab10 con 0x56209f2a87a0 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa420ff9640 10 client.15417 handle_command_reply: tid=0 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 client.15417 shutdown 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 -- 172.21.15.103:0/783127921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 msgr2=0x56209f294ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f294ce0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa42c02f840 tx=0x7fa42c02a950 comp rx=0 tx=0).stop 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 -- 172.21.15.103:0/783127921 shutdown_connections 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x56209f2a87a0 0x56209f2a9c10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f294ce0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 0x56209f2947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa420ff9640 0 client.15417 ms_handle_reset on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 --2- 172.21.15.103:0/783127921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f297450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 -- 172.21.15.103:0/783127921 >> 172.21.15.103:0/783127921 conn(0x56209f288f60 msgr2=0x56209f28ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 -- 172.21.15.103:0/783127921 shutdown_connections 2024-09-05T02:02:57.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 1 -- 172.21.15.103:0/783127921 wait complete. 2024-09-05T02:02:57.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:56.997+0000 7fa445a2f740 20 client.15417 trim_cache size 0 max 16384 2024-09-05T02:02:57.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.015+0000 7fa445a2f740 1 Processor -- start 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.015+0000 7fa445a2f740 1 -- start start 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28c9d0 0x56209f2a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28d2f0 0x56209f294760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 0x56209f294ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa445a2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56209f2a7f50 con 0x56209f28c9d0 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa445a2f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56209f2a8620 con 0x56209f28d2f0 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa445a2f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56209f2a8cf0 con 0x56209f297bb0 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa4413f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28c9d0 0x56209f2a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa4413f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28c9d0 0x56209f2a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32916/0 (socket says 172.21.15.103:32916) 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa4413f2640 1 -- 172.21.15.103:0/1622365892 learned_addr learned my addr 172.21.15.103:0/1622365892 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa441bf3640 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 0x56209f294ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa440bf1640 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28d2f0 0x56209f294760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa4413f2640 1 -- 172.21.15.103:0/1622365892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 msgr2=0x56209f294ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa4413f2640 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 0x56209f294ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa4413f2640 1 -- 172.21.15.103:0/1622365892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28d2f0 msgr2=0x56209f294760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa4413f2640 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28d2f0 0x56209f294760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa4413f2640 1 -- 172.21.15.103:0/1622365892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209f28d120 con 0x56209f28c9d0 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa441bf3640 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 0x56209f294ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa440bf1640 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28d2f0 0x56209f294760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa4413f2640 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28c9d0 0x56209f2a9c10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa434097820 tx=0x7fa434097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd00a292759a290 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.016+0000 7fa422ffd640 1 -- 172.21.15.103:0/1622365892 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434020040 con 0x56209f28c9d0 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa422ffd640 1 -- 172.21.15.103:0/1622365892 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa43409bac0 con 0x56209f28c9d0 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa422ffd640 1 -- 172.21.15.103:0/1622365892 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43409b3c0 con 0x56209f28c9d0 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa445a2f740 1 -- 172.21.15.103:0/1622365892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28c9d0 msgr2=0x56209f2a9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:57.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa445a2f740 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28c9d0 0x56209f2a9c10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa434097820 tx=0x7fa434097040 comp rx=0 tx=0).stop 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa445a2f740 1 -- 172.21.15.103:0/1622365892 shutdown_connections 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa445a2f740 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f297bb0 0x56209f294ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa445a2f740 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28d2f0 0x56209f294760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa445a2f740 1 --2- 172.21.15.103:0/1622365892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f28c9d0 0x56209f2a9c10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa445a2f740 1 -- 172.21.15.103:0/1622365892 >> 172.21.15.103:0/1622365892 conn(0x56209f288f60 msgr2=0x56209f283ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa445a2f740 1 -- 172.21.15.103:0/1622365892 shutdown_connections 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.017+0000 7fa445a2f740 1 -- 172.21.15.103:0/1622365892 wait complete. 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa445a2f740 1 -- start start 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa445a2f740 1 Processor -- start 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f2ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 0x56209f2ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa445a2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f2ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa445a2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56209f2a9a90 con 0x56209f297bb0 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa445a2f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56209f2a93c0 con 0x56209f28c9d0 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa445a2f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56209f2aa160 con 0x56209f28d2f0 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa445a2f740 10 client.0 _ll_register_callbacks cb 0x7fa42c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa441bf3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f2ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa441bf3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f2ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32922/0 (socket says 172.21.15.103:32922) 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa441bf3640 1 -- 172.21.15.103:0/2071071827 learned_addr learned my addr 172.21.15.103:0/2071071827 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa440bf1640 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 0x56209f2ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.018+0000 7fa4413f2640 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f2ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa441bf3640 1 -- 172.21.15.103:0/2071071827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 msgr2=0x56209f2ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa441bf3640 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 0x56209f2ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa441bf3640 1 -- 172.21.15.103:0/2071071827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 msgr2=0x56209f2ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa441bf3640 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f2ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa441bf3640 1 -- 172.21.15.103:0/2071071827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209f310ea0 con 0x56209f297bb0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa441bf3640 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f2ad8a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009890 tx=0x7fa42c030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4722813638194b2c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa420ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa420ff9640 1 -- 172.21.15.103:0/2071071827 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c04c040 con 0x56209f297bb0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa420ff9640 1 -- 172.21.15.103:0/2071071827 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa42c004250 con 0x56209f297bb0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa445a2f740 1 -- 172.21.15.103:0/2071071827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43409bac0 con 0x56209f297bb0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa420ff9640 1 -- 172.21.15.103:0/2071071827 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa42c06c050 con 0x56209f297bb0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa445a2f740 10 client.15423 fetch_fsmap learned FSMap version 29 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa445a2f740 1 -- 172.21.15.103:0/2071071827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56209f311fa0 con 0x56209f297bb0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa420ff9640 1 -- 172.21.15.103:0/2071071827 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa43409bac0 con 0x56209f297bb0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa420ff9640 1 -- 172.21.15.103:0/2071071827 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa42c051800 con 0x56209f297bb0 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.019+0000 7fa445a2f740 10 client.15423 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:02:57.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa445a2f740 10 client.15423 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa445a2f740 20 client.15423 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa445a2f740 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x56209f316b20 0x56209f315830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa445a2f740 4 client.15423 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa445a2f740 1 -- 172.21.15.103:0/2071071827 --> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] -- command(tid 0: {"prefix": "status"}) -- 0x56209f1770e0 con 0x56209f316b20 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa4413f2640 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x56209f316b20 0x56209f315830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa4413f2640 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x56209f316b20 0x56209f315830 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=874c0bd6860c85d4 server_cookie=578688cca4e5d2a3 in_seq=0 out_seq=0 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa420ff9640 10 client.15423 ms_handle_connect on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa420ff9640 1 -- 172.21.15.103:0/2071071827 <== mds.3 v2:172.21.15.167:6836/3468047781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56209f1770e0 con 0x56209f316b20 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa420ff9640 10 client.15423 handle_command_reply: tid=0 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.020+0000 7fa445a2f740 1 client.15423 shutdown 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 -- 172.21.15.103:0/2071071827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 msgr2=0x56209f2ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f2ad8a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009890 tx=0x7fa42c030650 comp rx=0 tx=0).stop 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 -- 172.21.15.103:0/2071071827 shutdown_connections 2024-09-05T02:02:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.167:6836/3468047781,v1:172.21.15.167:6837/3468047781] conn(0x56209f316b20 0x56209f315830 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56209f297bb0 0x56209f2ad8a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56209f28d2f0 0x56209f2ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 --2- 172.21.15.103:0/2071071827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56209f28c9d0 0x56209f2ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 -- 172.21.15.103:0/2071071827 >> 172.21.15.103:0/2071071827 conn(0x56209f288f60 msgr2=0x56209f294760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:57.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa420ff9640 0 client.15423 ms_handle_reset on v2:172.21.15.167:6836/3468047781 2024-09-05T02:02:57.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 -- 172.21.15.103:0/2071071827 shutdown_connections 2024-09-05T02:02:57.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 1 -- 172.21.15.103:0/2071071827 wait complete. 2024-09-05T02:02:57.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa445a2f740 20 client.15423 trim_cache size 0 max 16384 2024-09-05T02:02:57.034 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24457, 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 119.813970439, 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 29, 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 188.77839683400001 2024-09-05T02:02:57.035 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:57.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa443656640 1 -- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4240a2b80 msgr2=0x7fa4240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:57.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa443656640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4240a2b80 0x7fa4240a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009a60 tx=0x7fa42c0095c0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa443656640 1 -- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c100e20 msgr2=0x7fa43c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:57.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.021+0000 7fa443656640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c100e20 0x7fa43c0fcc80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa43400d770 tx=0x7fa43400dc30 comp rx=0 tx=0).stop 2024-09-05T02:02:57.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.022+0000 7fa443656640 1 -- 172.21.15.103:0/1191269288 shutdown_connections 2024-09-05T02:02:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.022+0000 7fa443656640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c102ef0 0x7fa43c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.022+0000 7fa443656640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4240a2b80 0x7fa4240a5040 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.022+0000 7fa443656640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa43c102570 0x7fa43c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.022+0000 7fa443656640 1 --2- 172.21.15.103:0/1191269288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa43c100e20 0x7fa43c0fcc80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.022+0000 7fa443656640 1 -- 172.21.15.103:0/1191269288 >> 172.21.15.103:0/1191269288 conn(0x7fa43c0fc7d0 msgr2=0x7fa43c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.022+0000 7fa443656640 1 -- 172.21.15.103:0/1191269288 shutdown_connections 2024-09-05T02:02:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:57.022+0000 7fa443656640 1 -- 172.21.15.103:0/1191269288 wait complete. 2024-09-05T02:02:57.829 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T02:02:58.076 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:02:58.573 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:02:58 smithi103 ceph-mon[30902]: pgmap v366: 129 pgs: 129 active+clean; 657 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-05T02:02:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:02:58 smithi167 ceph-mon[35122]: pgmap v366: 129 pgs: 129 active+clean; 657 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-05T02:02:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:02:58 smithi139 ceph-mon[34262]: pgmap v366: 129 pgs: 129 active+clean; 657 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-05T02:02:58.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.652+0000 7fbd9f359640 1 -- 172.21.15.103:0/2656718615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 msgr2=0x7fbd98100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:58.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.652+0000 7fbd9f359640 1 --2- 172.21.15.103:0/2656718615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 0x7fbd98100530 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009900 tx=0x7fbd8802a410 comp rx=0 tx=0).stop 2024-09-05T02:02:58.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.652+0000 7fbd9f359640 1 -- 172.21.15.103:0/2656718615 shutdown_connections 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.652+0000 7fbd9f359640 1 --2- 172.21.15.103:0/2656718615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd98100a70 0x7fbd9810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.652+0000 7fbd9f359640 1 --2- 172.21.15.103:0/2656718615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 0x7fbd98100530 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.652+0000 7fbd9f359640 1 --2- 172.21.15.103:0/2656718615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd98106ba0 0x7fbd98106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.652+0000 7fbd9f359640 1 -- 172.21.15.103:0/2656718615 >> 172.21.15.103:0/2656718615 conn(0x7fbd98077210 msgr2=0x7fbd980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.652+0000 7fbd9f359640 1 -- 172.21.15.103:0/2656718615 shutdown_connections 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.652+0000 7fbd9f359640 1 -- 172.21.15.103:0/2656718615 wait complete. 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9f359640 1 Processor -- start 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9f359640 1 -- start start 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9f359640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 0x7fbd98198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9f359640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd98100a70 0x7fbd98198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9f359640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd98106ba0 0x7fbd9819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9f359640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98111f40 con 0x7fbd981000d0 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9f359640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98111dc0 con 0x7fbd98100a70 2024-09-05T02:02:58.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9f359640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981120c0 con 0x7fbd98106ba0 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 0x7fbd98198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 0x7fbd98198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32944/0 (socket says 172.21.15.103:32944) 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/3724891144 learned_addr learned my addr 172.21.15.103:0/3724891144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9d8f6640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd98106ba0 0x7fbd9819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd98100a70 0x7fbd98198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/3724891144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd98106ba0 msgr2=0x7fbd9819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd98106ba0 0x7fbd9819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/3724891144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd98100a70 msgr2=0x7fbd98198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd98100a70 0x7fbd98198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.653+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/3724891144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9819d3d0 con 0x7fbd981000d0 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.654+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 0x7fbd98198420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d770 tx=0x7fbd9400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.654+0000 7fbd8e7fc640 1 -- 172.21.15.103:0/3724891144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94017070 con 0x7fbd981000d0 2024-09-05T02:02:58.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.654+0000 7fbd8e7fc640 1 -- 172.21.15.103:0/3724891144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd940054d0 con 0x7fbd981000d0 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.654+0000 7fbd9f359640 1 -- 172.21.15.103:0/3724891144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9819d6c0 con 0x7fbd981000d0 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.654+0000 7fbd8e7fc640 1 -- 172.21.15.103:0/3724891144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94004cf0 con 0x7fbd981000d0 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.654+0000 7fbd9f359640 1 -- 172.21.15.103:0/3724891144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981a4fa0 con 0x7fbd981000d0 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.656+0000 7fbd8e7fc640 1 -- 172.21.15.103:0/3724891144 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbd94004e90 con 0x7fbd981000d0 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.656+0000 7fbda1732740 1 Processor -- start 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.656+0000 7fbda1732740 1 -- start start 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbda1732740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e794f890 0x5589e794fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbda1732740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e79501b0 0x5589e795ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbda1732740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 0x5589e79613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbda1732740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589e78b6090 con 0x5589e79501b0 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbda1732740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5589e785da70 con 0x5589e794f890 2024-09-05T02:02:58.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbda1732740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5589e78b2050 con 0x5589e795efc0 2024-09-05T02:02:58.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd8e7fc640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbd840a2c00 0x7fbd840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e79501b0 0x5589e795ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e79501b0 0x5589e795ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32960/0 (socket says 172.21.15.103:32960) 2024-09-05T02:02:58.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1899626215 learned_addr learned my addr 172.21.15.103:0/1899626215 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:58.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d8f6640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbd840a2c00 0x7fbd840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d8f6640 1 --2- 172.21.15.103:0/1899626215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e794f890 0x5589e794fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/1899626215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 0x5589e79613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd8e7fc640 1 -- 172.21.15.103:0/3724891144 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbd940ced30 con 0x7fbd981000d0 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1899626215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 msgr2=0x5589e79613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/1899626215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 0x5589e79613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1899626215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e794f890 msgr2=0x5589e794fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/1899626215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e794f890 0x5589e794fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1899626215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589e786a130 con 0x5589e79501b0 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d8f6640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbd840a2c00 0x7fbd840a50c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd90005fd0 tx=0x7fbd90009de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.657+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/1899626215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e79501b0 0x5589e795ea80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbd94097c60 tx=0x7fbd94005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa8487111b889b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbd837fe640 1 -- 172.21.15.103:0/1899626215 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94020040 con 0x5589e79501b0 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbd837fe640 1 -- 172.21.15.103:0/1899626215 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd94009c30 con 0x5589e79501b0 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbd837fe640 1 -- 172.21.15.103:0/1899626215 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94099590 con 0x5589e79501b0 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbda1732740 1 -- 172.21.15.103:0/1899626215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e79501b0 msgr2=0x5589e795ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:58.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbda1732740 1 --2- 172.21.15.103:0/1899626215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e79501b0 0x5589e795ea80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbd94097c60 tx=0x7fbd94005730 comp rx=0 tx=0).stop 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbda1732740 1 -- 172.21.15.103:0/1899626215 shutdown_connections 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbda1732740 1 --2- 172.21.15.103:0/1899626215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 0x5589e79613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbda1732740 1 --2- 172.21.15.103:0/1899626215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e79501b0 0x5589e795ea80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbda1732740 1 --2- 172.21.15.103:0/1899626215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e794f890 0x5589e794fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbda1732740 1 -- 172.21.15.103:0/1899626215 >> 172.21.15.103:0/1899626215 conn(0x5589e794bf60 msgr2=0x5589e794c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbda1732740 1 -- 172.21.15.103:0/1899626215 shutdown_connections 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.658+0000 7fbda1732740 1 -- 172.21.15.103:0/1899626215 wait complete. 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbda1732740 1 -- start start 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbda1732740 1 Processor -- start 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbda1732740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e795e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbda1732740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 0x5589e795bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbda1732740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 0x5589e795c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbda1732740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589e7961b90 con 0x5589e794f890 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbda1732740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5589e7870f20 con 0x5589e795efc0 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbda1732740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5589e7961d10 con 0x5589e79501b0 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbda1732740 10 client.0 _ll_register_callbacks cb 0x5589e783a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e795e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e795e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32962/0 (socket says 172.21.15.103:32962) 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1458470490 learned_addr learned my addr 172.21.15.103:0/1458470490 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 0x5589e795bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.659+0000 7fbd9d8f6640 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 0x5589e795c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1458470490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 msgr2=0x5589e795bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 0x5589e795bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1458470490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 msgr2=0x5589e795c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:58.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 0x5589e795c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1458470490 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589e76272b0 con 0x5589e794f890 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 0x5589e795bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e795e8e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbd9409af90 tx=0x7fbd940a3f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bb569ab37a7829 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1458470490 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94020040 con 0x5589e794f890 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1458470490 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbd94099330 con 0x5589e794f890 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbda1732740 1 -- 172.21.15.103:0/1458470490 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589e78c3600 con 0x5589e794f890 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1458470490 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd94005180 con 0x5589e794f890 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbda1732740 10 client.15435 fetch_fsmap learned FSMap version 29 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbda1732740 1 -- 172.21.15.103:0/1458470490 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589e796d5a0 con 0x5589e794f890 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1458470490 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5589e78c3600 con 0x5589e794f890 2024-09-05T02:02:58.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1458470490 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbd9409f120 con 0x5589e794f890 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbda1732740 10 client.15435 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbda1732740 10 client.15435 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbda1732740 20 client.15435 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.660+0000 7fbda1732740 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x5589e796f690 0x5589e796e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.661+0000 7fbda1732740 4 client.15435 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.661+0000 7fbda1732740 1 -- 172.21.15.103:0/1458470490 --> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589e78bdb10 con 0x5589e796f690 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.661+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x5589e796f690 0x5589e796e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.661+0000 7fbd80ff9640 10 client.15435 ms_handle_connect on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.661+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x5589e796f690 0x5589e796e8f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=743ddd33c2154c83 server_cookie=c1c12fa3dd3bc6fe in_seq=0 out_seq=0 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1458470490 <== mds.2 v2:172.21.15.139:6836/2336570561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589e78bdb10 con 0x5589e796f690 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbd80ff9640 10 client.15435 handle_command_reply: tid=0 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 client.15435 shutdown 2024-09-05T02:02:58.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 -- 172.21.15.103:0/1458470490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 msgr2=0x5589e795e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e795e8e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbd9409af90 tx=0x7fbd940a3f30 comp rx=0 tx=0).stop 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 -- 172.21.15.103:0/1458470490 shutdown_connections 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 0x5589e795c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x5589e796f690 0x5589e796e8f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 0x5589e795bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 --2- 172.21.15.103:0/1458470490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e795e8e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbd80ff9640 0 client.15435 ms_handle_reset on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 -- 172.21.15.103:0/1458470490 >> 172.21.15.103:0/1458470490 conn(0x5589e794bf60 msgr2=0x5589e7951cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 -- 172.21.15.103:0/1458470490 shutdown_connections 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 1 -- 172.21.15.103:0/1458470490 wait complete. 2024-09-05T02:02:58.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.662+0000 7fbda1732740 20 client.15435 trim_cache size 0 max 16384 2024-09-05T02:02:58.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbda1732740 1 Processor -- start 2024-09-05T02:02:58.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbda1732740 1 -- start start 2024-09-05T02:02:58.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbda1732740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e796f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbda1732740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e79501b0 0x5589e7970b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbda1732740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 0x5589e796efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbda1732740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589e796ee40 con 0x5589e794f890 2024-09-05T02:02:58.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbda1732740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5589e796f510 con 0x5589e79501b0 2024-09-05T02:02:58.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbda1732740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5589e796fbe0 con 0x5589e795efc0 2024-09-05T02:02:58.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e796f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e796f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32964/0 (socket says 172.21.15.103:32964) 2024-09-05T02:02:58.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/2521829034 learned_addr learned my addr 172.21.15.103:0/2521829034 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:58.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbd9d8f6640 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 0x5589e796efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.680+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e79501b0 0x5589e7970b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/2521829034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 msgr2=0x5589e796efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:58.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 0x5589e796efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/2521829034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e79501b0 msgr2=0x5589e7970b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e79501b0 0x5589e7970b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/2521829034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589e795cce0 con 0x5589e794f890 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e79501b0 0x5589e7970b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e796f690 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbd940a3920 tx=0x7fbd94097660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982f28bc1f7ca3b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd82ffd640 1 -- 172.21.15.103:0/2521829034 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94014020 con 0x5589e794f890 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd82ffd640 1 -- 172.21.15.103:0/2521829034 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd9407c070 con 0x5589e794f890 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.681+0000 7fbd82ffd640 1 -- 172.21.15.103:0/2521829034 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94020070 con 0x5589e794f890 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 -- 172.21.15.103:0/2521829034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 msgr2=0x5589e796f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e796f690 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbd940a3920 tx=0x7fbd94097660 comp rx=0 tx=0).stop 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 -- 172.21.15.103:0/2521829034 shutdown_connections 2024-09-05T02:02:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e795efc0 0x5589e796efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e79501b0 0x5589e7970b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 --2- 172.21.15.103:0/2521829034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e796f690 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 -- 172.21.15.103:0/2521829034 >> 172.21.15.103:0/2521829034 conn(0x5589e794bf60 msgr2=0x5589e7956a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 -- 172.21.15.103:0/2521829034 shutdown_connections 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 -- 172.21.15.103:0/2521829034 wait complete. 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 -- start start 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 Processor -- start 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e7971d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 0x5589e79722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.682+0000 7fbda1732740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 0x5589e79727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbda1732740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589e7970980 con 0x5589e794f890 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbda1732740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5589e79702b0 con 0x5589e795efc0 2024-09-05T02:02:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbda1732740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5589e7971050 con 0x5589e79501b0 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbda1732740 10 client.0 _ll_register_callbacks cb 0x7fbd90012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e7971d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d0f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e7971d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32974/0 (socket says 172.21.15.103:32974) 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1051286530 learned_addr learned my addr 172.21.15.103:0/1051286530 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d8f6640 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 0x5589e79727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 0x5589e79722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1051286530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 msgr2=0x5589e79722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 0x5589e79722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1051286530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 msgr2=0x5589e79727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 0x5589e79727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d0f5640 1 -- 172.21.15.103:0/1051286530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589e795c550 con 0x5589e794f890 2024-09-05T02:02:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd9d0f5640 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e7971d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbd940cc7f0 tx=0x7fbd94005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7235dcd09392432a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1051286530 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94014030 con 0x5589e794f890 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1051286530 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbd9409f120 con 0x5589e794f890 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbda1732740 1 -- 172.21.15.103:0/1051286530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd9407c070 con 0x5589e794f890 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1051286530 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd940ad050 con 0x5589e794f890 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.683+0000 7fbda1732740 10 client.15441 fetch_fsmap learned FSMap version 29 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbda1732740 1 -- 172.21.15.103:0/1051286530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589e795c550 con 0x5589e794f890 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1051286530 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd940a7070 con 0x5589e794f890 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1051286530 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbd9409a6e0 con 0x5589e794f890 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbda1732740 10 client.15441 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbda1732740 10 client.15441 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbda1732740 20 client.15441 populate_metadata read hostname 'smithi103' 2024-09-05T02:02:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbda1732740 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x5589e79dd230 0x5589e79dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbda1732740 4 client.15441 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbda1732740 1 -- 172.21.15.103:0/1051286530 --> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] -- command(tid 0: {"prefix": "status"}) -- 0x5589e783a0e0 con 0x5589e79dd230 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.684+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x5589e79dd230 0x5589e79dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbd9c8f4640 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x5589e79dd230 0x5589e79dbf40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d6ffd89911eb0ce server_cookie=9ca107652d57800c in_seq=0 out_seq=0 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbd80ff9640 10 client.15441 ms_handle_connect on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbd80ff9640 1 -- 172.21.15.103:0/1051286530 <== mds.2 v2:172.21.15.139:6836/2336570561 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5589e783a0e0 con 0x5589e79dd230 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbd80ff9640 10 client.15441 handle_command_reply: tid=0 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 client.15441 shutdown 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 -- 172.21.15.103:0/1051286530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 msgr2=0x5589e7971d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e7971d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbd940cc7f0 tx=0x7fbd94005bb0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 -- 172.21.15.103:0/1051286530 shutdown_connections 2024-09-05T02:02:58.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5589e795efc0 0x5589e79727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.139:6836/2336570561,v1:172.21.15.139:6837/2336570561] conn(0x5589e79dd230 0x5589e79dbf40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589e79501b0 0x5589e79722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 --2- 172.21.15.103:0/1051286530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589e794f890 0x5589e7971d70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbd80ff9640 0 client.15441 ms_handle_reset on v2:172.21.15.139:6836/2336570561 2024-09-05T02:02:58.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 -- 172.21.15.103:0/1051286530 >> 172.21.15.103:0/1051286530 conn(0x5589e794bf60 msgr2=0x5589e7957240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:58.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 -- 172.21.15.103:0/1051286530 shutdown_connections 2024-09-05T02:02:58.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 1 -- 172.21.15.103:0/1051286530 wait complete. 2024-09-05T02:02:58.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.685+0000 7fbda1732740 20 client.15441 trim_cache size 0 max 16384 2024-09-05T02:02:58.699 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24464, 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 122.488568942, 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 29, 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 187.73168481799999 2024-09-05T02:02:58.700 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:02:58.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 -- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbd840a2c00 msgr2=0x7fbd840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:58.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbd840a2c00 0x7fbd840a50c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd90005fd0 tx=0x7fbd90009de0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 -- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 msgr2=0x7fbd98198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:02:58.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 0x7fbd98198420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d770 tx=0x7fbd9400dc30 comp rx=0 tx=0).stop 2024-09-05T02:02:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 -- 172.21.15.103:0/3724891144 shutdown_connections 2024-09-05T02:02:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fbd840a2c00 0x7fbd840a50c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd98106ba0 0x7fbd9819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd98100a70 0x7fbd98198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 --2- 172.21.15.103:0/3724891144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd981000d0 0x7fbd98198420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:02:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 -- 172.21.15.103:0/3724891144 >> 172.21.15.103:0/3724891144 conn(0x7fbd98077210 msgr2=0x7fbd980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:02:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 -- 172.21.15.103:0/3724891144 shutdown_connections 2024-09-05T02:02:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:02:58.686+0000 7fbd9f359640 1 -- 172.21.15.103:0/3724891144 wait complete. 2024-09-05T02:02:59.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T02:02:59.748 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.296+0000 7f4650226640 1 -- 172.21.15.103:0/1783157864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102ef0 msgr2=0x7f464810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.296+0000 7f4650226640 1 --2- 172.21.15.103:0/1783157864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102ef0 0x7f464810da40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4640009ca0 tx=0x7f464002a950 comp rx=0 tx=0).stop 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.296+0000 7f4650226640 1 -- 172.21.15.103:0/1783157864 shutdown_connections 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.296+0000 7f4650226640 1 --2- 172.21.15.103:0/1783157864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102ef0 0x7f464810da40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.296+0000 7f4650226640 1 --2- 172.21.15.103:0/1783157864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4648102570 0x7f46481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.296+0000 7f4650226640 1 --2- 172.21.15.103:0/1783157864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4648100e20 0x7f4648101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.296+0000 7f4650226640 1 -- 172.21.15.103:0/1783157864 >> 172.21.15.103:0/1783157864 conn(0x7f46480fc7d0 msgr2=0x7f46480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.296+0000 7f4650226640 1 -- 172.21.15.103:0/1783157864 shutdown_connections 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.296+0000 7f4650226640 1 -- 172.21.15.103:0/1783157864 wait complete. 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f4650226640 1 Processor -- start 2024-09-05T02:03:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f4650226640 1 -- start start 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f4650226640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4648100e20 0x7f46480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f4650226640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102570 0x7f46480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f4650226640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4648102ef0 0x7f46480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f4650226640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4648110e30 con 0x7f4648102570 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f4650226640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4648110cb0 con 0x7f4648102ef0 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f4650226640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4648110fb0 con 0x7f4648100e20 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464d7c1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102570 0x7f46480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464d7c1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102570 0x7f46480fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57802/0 (socket says 172.21.15.103:57802) 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464d7c1640 1 -- 172.21.15.103:0/3857469093 learned_addr learned my addr 172.21.15.103:0/3857469093 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464dfc2640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4648100e20 0x7f46480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464e7c3640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4648102ef0 0x7f46480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464d7c1640 1 -- 172.21.15.103:0/3857469093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4648100e20 msgr2=0x7f46480fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464d7c1640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4648100e20 0x7f46480fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464d7c1640 1 -- 172.21.15.103:0/3857469093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4648102ef0 msgr2=0x7f46480fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464d7c1640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4648102ef0 0x7f46480fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464d7c1640 1 -- 172.21.15.103:0/3857469093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46481a6840 con 0x7f4648102570 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.297+0000 7f464e7c3640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4648102ef0 0x7f46480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.298+0000 7f464d7c1640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102570 0x7f46480fd1c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d950 tx=0x7f463c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.298+0000 7f463affd640 1 -- 172.21.15.103:0/3857469093 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c017030 con 0x7f4648102570 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.298+0000 7f4650226640 1 -- 172.21.15.103:0/3857469093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46481a6b30 con 0x7f4648102570 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.298+0000 7f463affd640 1 -- 172.21.15.103:0/3857469093 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f463c004020 con 0x7f4648102570 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.298+0000 7f463affd640 1 -- 172.21.15.103:0/3857469093 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c0047a0 con 0x7f4648102570 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.298+0000 7f4650226640 1 -- 172.21.15.103:0/3857469093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46481a70c0 con 0x7f4648102570 2024-09-05T02:03:00.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.300+0000 7f463affd640 1 -- 172.21.15.103:0/3857469093 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f463c0041d0 con 0x7f4648102570 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.300+0000 7f46525ff740 1 Processor -- start 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.300+0000 7f46525ff740 1 -- start start 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f46525ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fa0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f46525ff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 0x558fb5fab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f46525ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fabbb0 0x558fb5fadfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f46525ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fb5f07090 con 0x558fb5fa09d0 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f46525ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558fb5eaea70 con 0x558fb5fabbb0 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f46525ff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558fb5f03050 con 0x558fb5fa12f0 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464e7c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fa0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464e7c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fa0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57812/0 (socket says 172.21.15.103:57812) 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464e7c3640 1 -- 172.21.15.103:0/2017046474 learned_addr learned my addr 172.21.15.103:0/2017046474 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f463affd640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46340a2b80 0x7f46340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464dfc2640 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 0x558fb5fab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464d7c1640 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fabbb0 0x558fb5fadfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f463affd640 1 -- 172.21.15.103:0/3857469093 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f463c0cee10 con 0x7f4648102570 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464e7c3640 1 -- 172.21.15.103:0/2017046474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 msgr2=0x558fb5fab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464e7c3640 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 0x558fb5fab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464e7c3640 1 -- 172.21.15.103:0/2017046474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fabbb0 msgr2=0x558fb5fadfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464e7c3640 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fabbb0 0x558fb5fadfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464e7c3640 1 -- 172.21.15.103:0/2017046474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fb5ebb130 con 0x558fb5fa09d0 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464dfc2640 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 0x558fb5fab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.301+0000 7f464e7c3640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46340a2b80 0x7f46340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.302+0000 7f464e7c3640 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fa0db0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4640004af0 tx=0x7f464002fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c073d444eaa4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.302+0000 7f4633fff640 1 -- 172.21.15.103:0/2017046474 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640034040 con 0x558fb5fa09d0 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.302+0000 7f4633fff640 1 -- 172.21.15.103:0/2017046474 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4640003d30 con 0x558fb5fa09d0 2024-09-05T02:03:00.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.302+0000 7f4633fff640 1 -- 172.21.15.103:0/2017046474 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46400397d0 con 0x558fb5fa09d0 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.302+0000 7f464e7c3640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46340a2b80 0x7f46340a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f464004c000 tx=0x7f4640049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 -- 172.21.15.103:0/2017046474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 msgr2=0x558fb5fa0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fa0db0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4640004af0 tx=0x7f464002fee0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 -- 172.21.15.103:0/2017046474 shutdown_connections 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fabbb0 0x558fb5fadfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 0x558fb5fab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 --2- 172.21.15.103:0/2017046474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fa0db0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 -- 172.21.15.103:0/2017046474 >> 172.21.15.103:0/2017046474 conn(0x558fb5f9cf60 msgr2=0x558fb5f9d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 -- 172.21.15.103:0/2017046474 shutdown_connections 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 -- 172.21.15.103:0/2017046474 wait complete. 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 -- start start 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.303+0000 7f46525ff740 1 Processor -- start 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f46525ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa09d0 0x558fb5fab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f46525ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa12f0 0x558fb5fa87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f46525ff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fabbb0 0x558fb5fa8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f46525ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fb5fae780 con 0x558fb5fa12f0 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f46525ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558fb5ec1f20 con 0x558fb5fa09d0 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f46525ff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558fb5fae900 con 0x558fb5fabbb0 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f46525ff740 10 client.0 _ll_register_callbacks cb 0x558fb5e8b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464d7c1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa12f0 0x558fb5fa87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464dfc2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa09d0 0x558fb5fab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464d7c1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa12f0 0x558fb5fa87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57814/0 (socket says 172.21.15.103:57814) 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464d7c1640 1 -- 172.21.15.103:0/1048687964 learned_addr learned my addr 172.21.15.103:0/1048687964 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464e7c3640 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fabbb0 0x558fb5fa8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464d7c1640 1 -- 172.21.15.103:0/1048687964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fabbb0 msgr2=0x558fb5fa8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:00.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464d7c1640 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fabbb0 0x558fb5fa8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464d7c1640 1 -- 172.21.15.103:0/1048687964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa09d0 msgr2=0x558fb5fab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464d7c1640 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa09d0 0x558fb5fab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464d7c1640 1 -- 172.21.15.103:0/1048687964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fb5c782b0 con 0x558fb5fa12f0 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464dfc2640 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa09d0 0x558fb5fab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f464d7c1640 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa12f0 0x558fb5fa87d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f463c097720 tx=0x7f463c0073b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499fb38781098dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.304+0000 7f46317fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46317fa640 1 -- 172.21.15.103:0/1048687964 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c0071c0 con 0x558fb5fa12f0 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46317fa640 1 -- 172.21.15.103:0/1048687964 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f463c017c10 con 0x558fb5fa12f0 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46525ff740 1 -- 172.21.15.103:0/1048687964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4640003d30 con 0x558fb5fa12f0 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46317fa640 1 -- 172.21.15.103:0/1048687964 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f463c09b2c0 con 0x558fb5fa12f0 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46525ff740 10 client.15453 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46525ff740 1 -- 172.21.15.103:0/1048687964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fb5fba0d0 con 0x558fb5fa12f0 2024-09-05T02:03:00.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46317fa640 1 -- 172.21.15.103:0/1048687964 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4640003d30 con 0x558fb5fa12f0 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46317fa640 1 -- 172.21.15.103:0/1048687964 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f463c050b60 con 0x558fb5fa12f0 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46525ff740 10 client.15453 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46525ff740 10 client.15453 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46525ff740 20 client.15453 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46525ff740 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x558fb5fbc690 0x558fb5fbdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46525ff740 4 client.15453 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.305+0000 7f46525ff740 1 -- 172.21.15.103:0/1048687964 --> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fb5f14600 con 0x558fb5fbc690 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.306+0000 7f464dfc2640 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x558fb5fbc690 0x558fb5fbdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.306+0000 7f464dfc2640 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x558fb5fbc690 0x558fb5fbdb00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95b5091b6a00198e server_cookie=75b36e8cd7721e7 in_seq=0 out_seq=0 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.306+0000 7f46317fa640 10 client.15453 ms_handle_connect on v2:172.21.15.167:6838/1858062899 2024-09-05T02:03:00.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46317fa640 1 -- 172.21.15.103:0/1048687964 <== mds.1 v2:172.21.15.167:6838/1858062899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fb5f14600 con 0x558fb5fbc690 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46317fa640 10 client.15453 handle_command_reply: tid=0 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 client.15453 shutdown 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 -- 172.21.15.103:0/1048687964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa12f0 msgr2=0x558fb5fa87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa12f0 0x558fb5fa87d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f463c097720 tx=0x7f463c0073b0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 -- 172.21.15.103:0/1048687964 shutdown_connections 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x558fb5fbc690 0x558fb5fbdb00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fabbb0 0x558fb5fa8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa12f0 0x558fb5fa87d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 --2- 172.21.15.103:0/1048687964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa09d0 0x558fb5fab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46317fa640 0 client.15453 ms_handle_reset on v2:172.21.15.167:6838/1858062899 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 -- 172.21.15.103:0/1048687964 >> 172.21.15.103:0/1048687964 conn(0x558fb5f9cf60 msgr2=0x558fb5fa2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 -- 172.21.15.103:0/1048687964 shutdown_connections 2024-09-05T02:03:00.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 1 -- 172.21.15.103:0/1048687964 wait complete. 2024-09-05T02:03:00.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.307+0000 7f46525ff740 20 client.15453 trim_cache size 0 max 16384 2024-09-05T02:03:00.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.325+0000 7f46525ff740 1 Processor -- start 2024-09-05T02:03:00.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.325+0000 7f46525ff740 1 -- start start 2024-09-05T02:03:00.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.325+0000 7f46525ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fbdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.325+0000 7f46525ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa12f0 0x558fb5f97ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.325+0000 7f46525ff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fabbb0 0x558fb5fa70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.325+0000 7f46525ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fb5fbbe40 con 0x558fb5fa09d0 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.325+0000 7f46525ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558fb5fbc510 con 0x558fb5fa12f0 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.325+0000 7f46525ff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558fb5fbcbe0 con 0x558fb5fabbb0 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f464dfc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fbdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f464dfc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fbdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57818/0 (socket says 172.21.15.103:57818) 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f464dfc2640 1 -- 172.21.15.103:0/878455592 learned_addr learned my addr 172.21.15.103:0/878455592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f464dfc2640 1 -- 172.21.15.103:0/878455592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fabbb0 msgr2=0x558fb5fa70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f464dfc2640 1 --2- 172.21.15.103:0/878455592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fabbb0 0x558fb5fa70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f464dfc2640 1 -- 172.21.15.103:0/878455592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa12f0 msgr2=0x558fb5f97ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f464dfc2640 1 --2- 172.21.15.103:0/878455592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa12f0 0x558fb5f97ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f464dfc2640 1 -- 172.21.15.103:0/878455592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fb5fab0a0 con 0x558fb5fa09d0 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f464dfc2640 1 --2- 172.21.15.103:0/878455592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fbdb00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4644005fd0 tx=0x7f464400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4da76ec923a8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f46337fe640 1 -- 172.21.15.103:0/878455592 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644005020 con 0x558fb5fa09d0 2024-09-05T02:03:00.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f46337fe640 1 -- 172.21.15.103:0/878455592 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46440047a0 con 0x558fb5fa09d0 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.326+0000 7f46337fe640 1 -- 172.21.15.103:0/878455592 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46440116a0 con 0x558fb5fa09d0 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 -- 172.21.15.103:0/878455592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 msgr2=0x558fb5fbdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 --2- 172.21.15.103:0/878455592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fbdb00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4644005fd0 tx=0x7f464400b390 comp rx=0 tx=0).stop 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 -- 172.21.15.103:0/878455592 shutdown_connections 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 --2- 172.21.15.103:0/878455592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fabbb0 0x558fb5fa70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 --2- 172.21.15.103:0/878455592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa12f0 0x558fb5f97ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 --2- 172.21.15.103:0/878455592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fa09d0 0x558fb5fbdb00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 -- 172.21.15.103:0/878455592 >> 172.21.15.103:0/878455592 conn(0x558fb5f9cf60 msgr2=0x558fb5fa8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 -- 172.21.15.103:0/878455592 shutdown_connections 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 -- 172.21.15.103:0/878455592 wait complete. 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 -- start start 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.327+0000 7f46525ff740 1 Processor -- start 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f46525ff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 0x558fb5fc60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f46525ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fabbb0 0x558fb5fc65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f46525ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa5160 0x558fb5fa55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f46525ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fb5fbd980 con 0x558fb5fabbb0 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f46525ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558fb5fbd2b0 con 0x558fb5fa5160 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f46525ff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558fb5fbe050 con 0x558fb5fa12f0 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f46525ff740 10 client.0 _ll_register_callbacks cb 0x7f463c005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464d7c1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fabbb0 0x558fb5fc65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464d7c1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fabbb0 0x558fb5fc65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57832/0 (socket says 172.21.15.103:57832) 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464d7c1640 1 -- 172.21.15.103:0/4019194604 learned_addr learned my addr 172.21.15.103:0/4019194604 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464dfc2640 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 0x558fb5fc60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464e7c3640 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa5160 0x558fb5fa55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464d7c1640 1 -- 172.21.15.103:0/4019194604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 msgr2=0x558fb5fc60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464d7c1640 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 0x558fb5fc60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464d7c1640 1 -- 172.21.15.103:0/4019194604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa5160 msgr2=0x558fb5fa55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:00.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464d7c1640 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa5160 0x558fb5fa55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464d7c1640 1 -- 172.21.15.103:0/4019194604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fb5fa5df0 con 0x558fb5fabbb0 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464e7c3640 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa5160 0x558fb5fa55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f464d7c1640 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fabbb0 0x558fb5fc65f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f463c09afc0 tx=0x7f463c050e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6be81fd2f99c8f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.328+0000 7f46317fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46317fa640 1 -- 172.21.15.103:0/4019194604 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c014070 con 0x558fb5fabbb0 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46317fa640 1 -- 172.21.15.103:0/4019194604 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f463c09a970 con 0x558fb5fabbb0 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46525ff740 1 -- 172.21.15.103:0/4019194604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46440047a0 con 0x558fb5fabbb0 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46317fa640 1 -- 172.21.15.103:0/4019194604 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f463c0071c0 con 0x558fb5fabbb0 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46525ff740 10 client.15459 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46525ff740 1 -- 172.21.15.103:0/4019194604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fb6026da0 con 0x558fb5fabbb0 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46317fa640 1 -- 172.21.15.103:0/4019194604 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46440047a0 con 0x558fb5fabbb0 2024-09-05T02:03:00.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46317fa640 1 -- 172.21.15.103:0/4019194604 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f463c009a30 con 0x558fb5fabbb0 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46525ff740 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46525ff740 10 client.15459 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46525ff740 20 client.15459 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46525ff740 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x558fb602b6a0 0x558fb602a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46525ff740 4 client.15459 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.329+0000 7f46525ff740 1 -- 172.21.15.103:0/4019194604 --> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] -- command(tid 0: {"prefix": "status"}) -- 0x558fb5f0eb10 con 0x558fb602b6a0 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f464dfc2640 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x558fb602b6a0 0x558fb602a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46317fa640 10 client.15459 ms_handle_connect on v2:172.21.15.167:6838/1858062899 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f464dfc2640 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x558fb602b6a0 0x558fb602a3b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=244af7f65a8f37a2 server_cookie=6b540f0409fd197e in_seq=0 out_seq=0 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46317fa640 1 -- 172.21.15.103:0/4019194604 <== mds.1 v2:172.21.15.167:6838/1858062899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558fb5f0eb10 con 0x558fb602b6a0 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46317fa640 10 client.15459 handle_command_reply: tid=0 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 client.15459 shutdown 2024-09-05T02:03:00.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 -- 172.21.15.103:0/4019194604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fabbb0 msgr2=0x558fb5fc65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fabbb0 0x558fb5fc65f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f463c09afc0 tx=0x7f463c050e90 comp rx=0 tx=0).stop 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 -- 172.21.15.103:0/4019194604 shutdown_connections 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.167:6838/1858062899,v1:172.21.15.167:6839/1858062899] conn(0x558fb602b6a0 0x558fb602a3b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558fb5fa5160 0x558fb5fa55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fb5fabbb0 0x558fb5fc65f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46317fa640 0 client.15459 ms_handle_reset on v2:172.21.15.167:6838/1858062899 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 --2- 172.21.15.103:0/4019194604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558fb5fa12f0 0x558fb5fc60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 -- 172.21.15.103:0/4019194604 >> 172.21.15.103:0/4019194604 conn(0x558fb5f9cf60 msgr2=0x558fb5fa2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.330+0000 7f46525ff740 1 -- 172.21.15.103:0/4019194604 shutdown_connections 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f46525ff740 1 -- 172.21.15.103:0/4019194604 wait complete. 2024-09-05T02:03:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f46525ff740 20 client.15459 trim_cache size 0 max 16384 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24466, 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 125.138509977, 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 29, 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T02:03:00.340 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 186.77316788900001 2024-09-05T02:03:00.341 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:03:00.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 -- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46340a2b80 msgr2=0x7f46340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:00.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46340a2b80 0x7f46340a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f464004c000 tx=0x7f4640049000 comp rx=0 tx=0).stop 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 -- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102570 msgr2=0x7f46480fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102570 0x7f46480fd1c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d950 tx=0x7f463c00de10 comp rx=0 tx=0).stop 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 -- 172.21.15.103:0/3857469093 shutdown_connections 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4648102ef0 0x7f46480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4648102570 0x7f46480fd1c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f46340a2b80 0x7f46340a5040 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 --2- 172.21.15.103:0/3857469093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4648100e20 0x7f46480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 -- 172.21.15.103:0/3857469093 >> 172.21.15.103:0/3857469093 conn(0x7f46480fc7d0 msgr2=0x7f464806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 -- 172.21.15.103:0/3857469093 shutdown_connections 2024-09-05T02:03:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:00.331+0000 7f4650226640 1 -- 172.21.15.103:0/3857469093 wait complete. 2024-09-05T02:03:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:00 smithi167 ceph-mon[35122]: pgmap v367: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-05T02:03:00.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:00 smithi139 ceph-mon[34262]: pgmap v367: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-05T02:03:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:00 smithi103 ceph-mon[30902]: pgmap v367: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-05T02:03:01.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T02:03:01.483 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:03:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:01.999+0000 7fca5856a640 1 -- 172.21.15.103:0/1759662032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50100a80 msgr2=0x7fca50100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:01.999+0000 7fca5856a640 1 --2- 172.21.15.103:0/1759662032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50100a80 0x7fca50100ee0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fca40009930 tx=0x7fca4002a420 comp rx=0 tx=0).stop 2024-09-05T02:03:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:01.999+0000 7fca5856a640 1 -- 172.21.15.103:0/1759662032 shutdown_connections 2024-09-05T02:03:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:01.999+0000 7fca5856a640 1 --2- 172.21.15.103:0/1759662032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca50101420 0x7fca5010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:01.999+0000 7fca5856a640 1 --2- 172.21.15.103:0/1759662032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50100a80 0x7fca50100ee0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:01.999+0000 7fca5856a640 1 --2- 172.21.15.103:0/1759662032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca50106a80 0x7fca50106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:01.999+0000 7fca5856a640 1 -- 172.21.15.103:0/1759662032 >> 172.21.15.103:0/1759662032 conn(0x7fca500fc7a0 msgr2=0x7fca500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:01.999+0000 7fca5856a640 1 -- 172.21.15.103:0/1759662032 shutdown_connections 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:01.999+0000 7fca5856a640 1 -- 172.21.15.103:0/1759662032 wait complete. 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca5856a640 1 Processor -- start 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca5856a640 1 -- start start 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca5856a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca50100a80 0x7fca50199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca5856a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca50101420 0x7fca50199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca5856a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50106a80 0x7fca501a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca5856a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca50110d30 con 0x7fca50106a80 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca5856a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca50110bb0 con 0x7fca50100a80 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca5856a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca50110eb0 con 0x7fca50101420 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56b07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50106a80 0x7fca501a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56b07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50106a80 0x7fca501a42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57858/0 (socket says 172.21.15.103:57858) 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56b07640 1 -- 172.21.15.103:0/551846530 learned_addr learned my addr 172.21.15.103:0/551846530 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca55b05640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca50101420 0x7fca50199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56306640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca50100a80 0x7fca50199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56b07640 1 -- 172.21.15.103:0/551846530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca50101420 msgr2=0x7fca50199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56b07640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca50101420 0x7fca50199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56b07640 1 -- 172.21.15.103:0/551846530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca50100a80 msgr2=0x7fca50199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56b07640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca50100a80 0x7fca50199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56b07640 1 -- 172.21.15.103:0/551846530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca501a47e0 con 0x7fca50106a80 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.000+0000 7fca56b07640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50106a80 0x7fca501a42a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fca4800eaa0 tx=0x7fca4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.001+0000 7fca477fe640 1 -- 172.21.15.103:0/551846530 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4800cdb0 con 0x7fca50106a80 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.001+0000 7fca5856a640 1 -- 172.21.15.103:0/551846530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca501a49b0 con 0x7fca50106a80 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.001+0000 7fca477fe640 1 -- 172.21.15.103:0/551846530 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca48005020 con 0x7fca50106a80 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.001+0000 7fca477fe640 1 -- 172.21.15.103:0/551846530 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480116d0 con 0x7fca50106a80 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.001+0000 7fca5856a640 1 -- 172.21.15.103:0/551846530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a4e70 con 0x7fca50106a80 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.003+0000 7fca477fe640 1 -- 172.21.15.103:0/551846530 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fca480051d0 con 0x7fca50106a80 2024-09-05T02:03:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.003+0000 7fca5a943740 1 Processor -- start 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.003+0000 7fca5a943740 1 -- start start 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.003+0000 7fca5a943740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e549db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.003+0000 7fca5a943740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 0x555b6e554670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.003+0000 7fca5a943740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e556fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.003+0000 7fca5a943740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b6e4b0090 con 0x555b6e554bb0 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.003+0000 7fca5a943740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555b6e457a70 con 0x555b6e5499d0 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.003+0000 7fca5a943740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b6e4ac050 con 0x555b6e54a2f0 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca55b05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e556fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca55b05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e556fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57862/0 (socket says 172.21.15.103:57862) 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca55b05640 1 -- 172.21.15.103:0/2050914023 learned_addr learned my addr 172.21.15.103:0/2050914023 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca56306640 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 0x555b6e554670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca55b05640 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e549db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca477fe640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca3c0a2b80 0x7fca3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca56b07640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca3c0a2b80 0x7fca3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca56306640 1 -- 172.21.15.103:0/2050914023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 msgr2=0x555b6e549db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca56306640 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e549db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca56306640 1 -- 172.21.15.103:0/2050914023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 msgr2=0x555b6e556fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca56306640 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e556fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca56306640 1 -- 172.21.15.103:0/2050914023 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6e464130 con 0x555b6e54a2f0 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca55b05640 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e556fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca477fe640 1 -- 172.21.15.103:0/551846530 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fca480cf6a0 con 0x7fca50106a80 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca56306640 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 0x555b6e554670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009a40 tx=0x7fca4c009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4d5b05aff064193 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca56b07640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca3c0a2b80 0x7fca3c0a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fca48097fb0 tx=0x7fca48093040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.004+0000 7fca44ff9640 1 -- 172.21.15.103:0/2050914023 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c016070 con 0x555b6e54a2f0 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca44ff9640 1 -- 172.21.15.103:0/2050914023 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca4c0054d0 con 0x555b6e54a2f0 2024-09-05T02:03:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca44ff9640 1 -- 172.21.15.103:0/2050914023 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c004d10 con 0x555b6e54a2f0 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca5a943740 1 -- 172.21.15.103:0/2050914023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 msgr2=0x555b6e554670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca5a943740 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 0x555b6e554670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009a40 tx=0x7fca4c009f00 comp rx=0 tx=0).stop 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca5a943740 1 -- 172.21.15.103:0/2050914023 shutdown_connections 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca5a943740 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e556fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca5a943740 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 0x555b6e554670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca5a943740 1 --2- 172.21.15.103:0/2050914023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e549db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca5a943740 1 -- 172.21.15.103:0/2050914023 >> 172.21.15.103:0/2050914023 conn(0x555b6e545f60 msgr2=0x555b6e546340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca5a943740 1 -- 172.21.15.103:0/2050914023 shutdown_connections 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.005+0000 7fca5a943740 1 -- 172.21.15.103:0/2050914023 wait complete. 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca5a943740 1 -- start start 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca5a943740 1 Processor -- start 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca5a943740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e5543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca5a943740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 0x555b6e551710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca5a943740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e551c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca5a943740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b6e557780 con 0x555b6e554bb0 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca5a943740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555b6e46af20 con 0x555b6e5499d0 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca5a943740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b6e557900 con 0x555b6e54a2f0 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca5a943740 10 client.0 _ll_register_callbacks cb 0x555b6e434030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca55b05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e551c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca55b05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e551c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57868/0 (socket says 172.21.15.103:57868) 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca55b05640 1 -- 172.21.15.103:0/296467873 learned_addr learned my addr 172.21.15.103:0/296467873 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca56306640 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 0x555b6e551710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca55b05640 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e5543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca55b05640 1 -- 172.21.15.103:0/296467873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 msgr2=0x555b6e551710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.006+0000 7fca55b05640 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 0x555b6e551710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca55b05640 1 -- 172.21.15.103:0/296467873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 msgr2=0x555b6e5543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca55b05640 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e5543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca55b05640 1 -- 172.21.15.103:0/296467873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6e2212b0 con 0x555b6e554bb0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca55b05640 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e551c50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fca40006010 tx=0x7fca40040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbc4f092ae32190 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca39ffb640 1 -- 172.21.15.103:0/296467873 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40035040 con 0x555b6e554bb0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca39ffb640 1 -- 172.21.15.103:0/296467873 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fca400315e0 con 0x555b6e554bb0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca5a943740 1 -- 172.21.15.103:0/296467873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca4c0054d0 con 0x555b6e554bb0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca39ffb640 1 -- 172.21.15.103:0/296467873 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca40053050 con 0x555b6e554bb0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca5a943740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca5a943740 1 -- 172.21.15.103:0/296467873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b6e563040 con 0x555b6e554bb0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca39ffb640 1 -- 172.21.15.103:0/296467873 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca4c0054d0 con 0x555b6e554bb0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca39ffb640 1 -- 172.21.15.103:0/296467873 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fca4003a040 con 0x555b6e554bb0 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca5a943740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca5a943740 10 client.15471 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T02:03:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca5a943740 20 client.15471 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca5a943740 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555b6e565610 0x555b6e566a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca5a943740 4 client.15471 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.007+0000 7fca5a943740 1 -- 172.21.15.103:0/296467873 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b6e4bd600 con 0x555b6e565610 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.008+0000 7fca56306640 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555b6e565610 0x555b6e566a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.008+0000 7fca56306640 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555b6e565610 0x555b6e566a80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2974428d79aa5339 server_cookie=29975bf8f9d1aef0 in_seq=0 out_seq=0 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.008+0000 7fca39ffb640 10 client.15471 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca39ffb640 1 -- 172.21.15.103:0/296467873 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b6e4bd600 con 0x555b6e565610 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca39ffb640 10 client.15471 handle_command_reply: tid=0 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 client.15471 shutdown 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 -- 172.21.15.103:0/296467873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 msgr2=0x555b6e551c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e551c50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fca40006010 tx=0x7fca40040040 comp rx=0 tx=0).stop 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 -- 172.21.15.103:0/296467873 shutdown_connections 2024-09-05T02:03:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555b6e565610 0x555b6e566a80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e554bb0 0x555b6e551c50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e54a2f0 0x555b6e551710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca39ffb640 0 client.15471 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 --2- 172.21.15.103:0/296467873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e5543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 -- 172.21.15.103:0/296467873 >> 172.21.15.103:0/296467873 conn(0x555b6e545f60 msgr2=0x555b6e54bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 -- 172.21.15.103:0/296467873 shutdown_connections 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 1 -- 172.21.15.103:0/296467873 wait complete. 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.009+0000 7fca5a943740 20 client.15471 trim_cache size 0 max 16384 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.027+0000 7fca5a943740 1 Processor -- start 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.027+0000 7fca5a943740 1 -- start start 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.027+0000 7fca5a943740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e564f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.027+0000 7fca5a943740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e54a2f0 0x555b6e566a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.027+0000 7fca5a943740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 0x555b6e550fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.027+0000 7fca5a943740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b6e564dc0 con 0x555b6e54a2f0 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.027+0000 7fca5a943740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555b6e565490 con 0x555b6e5499d0 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.027+0000 7fca5a943740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b6e565b60 con 0x555b6e554bb0 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.027+0000 7fca56306640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e54a2f0 0x555b6e566a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca56306640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e54a2f0 0x555b6e566a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57874/0 (socket says 172.21.15.103:57874) 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca56306640 1 -- 172.21.15.103:0/2549987690 learned_addr learned my addr 172.21.15.103:0/2549987690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca55b05640 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 0x555b6e550fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca55b05640 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e564f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca56306640 1 -- 172.21.15.103:0/2549987690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 msgr2=0x555b6e550fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca56306640 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 0x555b6e550fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca56306640 1 -- 172.21.15.103:0/2549987690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 msgr2=0x555b6e564f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca56306640 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e564f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca56306640 1 -- 172.21.15.103:0/2549987690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6e554730 con 0x555b6e54a2f0 2024-09-05T02:03:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca55b05640 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 0x555b6e550fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca56306640 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e54a2f0 0x555b6e566a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x555b6e546dc0 tx=0x7fca4c004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ebff2e71cde992 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca3bfff640 1 -- 172.21.15.103:0/2549987690 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c016040 con 0x555b6e54a2f0 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca3bfff640 1 -- 172.21.15.103:0/2549987690 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca4c00ec50 con 0x555b6e54a2f0 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.028+0000 7fca3bfff640 1 -- 172.21.15.103:0/2549987690 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c028690 con 0x555b6e54a2f0 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 -- 172.21.15.103:0/2549987690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e54a2f0 msgr2=0x555b6e566a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e54a2f0 0x555b6e566a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x555b6e546dc0 tx=0x7fca4c004990 comp rx=0 tx=0).stop 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 -- 172.21.15.103:0/2549987690 shutdown_connections 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 0x555b6e550fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e54a2f0 0x555b6e566a80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 --2- 172.21.15.103:0/2549987690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e5499d0 0x555b6e564f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 -- 172.21.15.103:0/2549987690 >> 172.21.15.103:0/2549987690 conn(0x555b6e545f60 msgr2=0x555b6e546340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 -- 172.21.15.103:0/2549987690 shutdown_connections 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 -- 172.21.15.103:0/2549987690 wait complete. 2024-09-05T02:03:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 -- start start 2024-09-05T02:03:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 Processor -- start 2024-09-05T02:03:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e5499d0 0x555b6e569e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e54a2f0 0x555b6e56a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 0x555b6e5cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b6e566900 con 0x555b6e5499d0 2024-09-05T02:03:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555b6e566230 con 0x555b6e54a2f0 2024-09-05T02:03:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b6e566fd0 con 0x555b6e554bb0 2024-09-05T02:03:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca5a943740 10 client.0 _ll_register_callbacks cb 0x7fca40005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca55b05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e5499d0 0x555b6e569e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca55b05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e5499d0 0x555b6e569e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57886/0 (socket says 172.21.15.103:57886) 2024-09-05T02:03:02.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.029+0000 7fca55b05640 1 -- 172.21.15.103:0/2806693073 learned_addr learned my addr 172.21.15.103:0/2806693073 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:02.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca56306640 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e54a2f0 0x555b6e56a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca55b05640 1 -- 172.21.15.103:0/2806693073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 msgr2=0x555b6e5cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:02.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca55b05640 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 0x555b6e5cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca55b05640 1 -- 172.21.15.103:0/2806693073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e54a2f0 msgr2=0x555b6e56a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:02.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca55b05640 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e54a2f0 0x555b6e56a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca55b05640 1 -- 172.21.15.103:0/2806693073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6e56ac60 con 0x555b6e5499d0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca56306640 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e54a2f0 0x555b6e56a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca55b05640 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e5499d0 0x555b6e569e30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fca40006010 tx=0x7fca40032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70e6a26ca7f6d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca39ffb640 1 -- 172.21.15.103:0/2806693073 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40035070 con 0x555b6e5499d0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca39ffb640 1 -- 172.21.15.103:0/2806693073 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fca400315e0 con 0x555b6e5499d0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca5a943740 1 -- 172.21.15.103:0/2806693073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca4c00ec50 con 0x555b6e5499d0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca39ffb640 1 -- 172.21.15.103:0/2806693073 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca4005f050 con 0x555b6e5499d0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca5a943740 10 client.15477 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca5a943740 1 -- 172.21.15.103:0/2806693073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b6e5cf2f0 con 0x555b6e5499d0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.030+0000 7fca39ffb640 1 -- 172.21.15.103:0/2806693073 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca4c00ec50 con 0x555b6e5499d0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca39ffb640 1 -- 172.21.15.103:0/2806693073 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fca4004db20 con 0x555b6e5499d0 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca5a943740 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:02.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca5a943740 10 client.15477 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca5a943740 20 client.15477 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca5a943740 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555b6e565610 0x555b6e5d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca5a943740 4 client.15477 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca5a943740 1 -- 172.21.15.103:0/2806693073 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "status"}) -- 0x555b6e4b7b10 con 0x555b6e565610 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca56306640 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555b6e565610 0x555b6e5d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca56306640 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555b6e565610 0x555b6e5d2c20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7330f379d8bf8dab server_cookie=ea9efb5bac983dbf in_seq=0 out_seq=0 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.031+0000 7fca39ffb640 10 client.15477 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca39ffb640 1 -- 172.21.15.103:0/2806693073 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555b6e4b7b10 con 0x555b6e565610 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca39ffb640 10 client.15477 handle_command_reply: tid=0 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 client.15477 shutdown 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 -- 172.21.15.103:0/2806693073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e5499d0 msgr2=0x555b6e569e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e5499d0 0x555b6e569e30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fca40006010 tx=0x7fca40032f00 comp rx=0 tx=0).stop 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 -- 172.21.15.103:0/2806693073 shutdown_connections 2024-09-05T02:03:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555b6e565610 0x555b6e5d2c20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e554bb0 0x555b6e5cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555b6e54a2f0 0x555b6e56a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 --2- 172.21.15.103:0/2806693073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b6e5499d0 0x555b6e569e30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 -- 172.21.15.103:0/2806693073 >> 172.21.15.103:0/2806693073 conn(0x555b6e545f60 msgr2=0x555b6e550f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca39ffb640 0 client.15477 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 -- 172.21.15.103:0/2806693073 shutdown_connections 2024-09-05T02:03:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 1 -- 172.21.15.103:0/2806693073 wait complete. 2024-09-05T02:03:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.032+0000 7fca5a943740 20 client.15477 trim_cache size 0 max 16384 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "695e8e42-6b29-11ef-bcd6-c7b262605968", 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24497, 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 166.66441001000001, 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 29, 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 90, 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 178.35001757699999 2024-09-05T02:03:02.065 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 -- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca3c0a2b80 msgr2=0x7fca3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca3c0a2b80 0x7fca3c0a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fca48097fb0 tx=0x7fca48093040 comp rx=0 tx=0).stop 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 -- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50106a80 msgr2=0x7fca501a42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50106a80 0x7fca501a42a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fca4800eaa0 tx=0x7fca4800ef60 comp rx=0 tx=0).stop 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 -- 172.21.15.103:0/551846530 shutdown_connections 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50106a80 0x7fca501a42a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca3c0a2b80 0x7fca3c0a5040 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca50101420 0x7fca50199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 --2- 172.21.15.103:0/551846530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca50100a80 0x7fca50199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 -- 172.21.15.103:0/551846530 >> 172.21.15.103:0/551846530 conn(0x7fca500fc7a0 msgr2=0x7fca500fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:02.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 -- 172.21.15.103:0/551846530 shutdown_connections 2024-09-05T02:03:02.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:02.033+0000 7fca5856a640 1 -- 172.21.15.103:0/551846530 wait complete. 2024-09-05T02:03:02.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:02 smithi167 ceph-mon[35122]: pgmap v368: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-09-05T02:03:02.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:02 smithi139 ceph-mon[34262]: pgmap v368: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-09-05T02:03:02.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:02 smithi103 ceph-mon[30902]: pgmap v368: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-09-05T02:03:02.760 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-05T02:03:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:04 smithi167 ceph-mon[35122]: pgmap v369: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-05T02:03:04.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:04 smithi139 ceph-mon[34262]: pgmap v369: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-05T02:03:04.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:04 smithi103 ceph-mon[30902]: pgmap v369: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-05T02:03:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:06 smithi167 ceph-mon[35122]: pgmap v370: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.2 KiB/s wr, 17 op/s 2024-09-05T02:03:06.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:06 smithi139 ceph-mon[34262]: pgmap v370: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.2 KiB/s wr, 17 op/s 2024-09-05T02:03:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:06 smithi103 ceph-mon[30902]: pgmap v370: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.2 KiB/s wr, 17 op/s 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.555+0000 7f9882aba640 1 -- 172.21.15.103:0/1371560252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c100a50 msgr2=0x7f987c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.555+0000 7f9882aba640 1 --2- 172.21.15.103:0/1371560252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c100a50 0x7f987c100e50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9870009900 tx=0x7f987002a410 comp rx=0 tx=0).stop 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.555+0000 7f9882aba640 1 -- 172.21.15.103:0/1371560252 shutdown_connections 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.555+0000 7f9882aba640 1 --2- 172.21.15.103:0/1371560252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f987c102600 0x7f987c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.555+0000 7f9882aba640 1 --2- 172.21.15.103:0/1371560252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f987c101c40 0x7f987c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.555+0000 7f9882aba640 1 --2- 172.21.15.103:0/1371560252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c100a50 0x7f987c100e50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.555+0000 7f9882aba640 1 -- 172.21.15.103:0/1371560252 >> 172.21.15.103:0/1371560252 conn(0x7f987c0fc1b0 msgr2=0x7f987c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.555+0000 7f9882aba640 1 -- 172.21.15.103:0/1371560252 shutdown_connections 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.555+0000 7f9882aba640 1 -- 172.21.15.103:0/1371560252 wait complete. 2024-09-05T02:03:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9882aba640 1 Processor -- start 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9882aba640 1 -- start start 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9882aba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f987c100a50 0x7f987c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9882aba640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f987c101c40 0x7f987c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9882aba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c102600 0x7f987c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9882aba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10d6f0 con 0x7f987c102600 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9882aba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10d570 con 0x7f987c100a50 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9882aba640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10d870 con 0x7f987c101c40 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9881057640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c102600 0x7f987c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9881057640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c102600 0x7f987c19f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57896/0 (socket says 172.21.15.103:57896) 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9881057640 1 -- 172.21.15.103:0/1312435332 learned_addr learned my addr 172.21.15.103:0/1312435332 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9880856640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f987c100a50 0x7f987c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f987bfff640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f987c101c40 0x7f987c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9881057640 1 -- 172.21.15.103:0/1312435332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f987c101c40 msgr2=0x7f987c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9881057640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f987c101c40 0x7f987c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9881057640 1 -- 172.21.15.103:0/1312435332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f987c100a50 msgr2=0x7f987c197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9881057640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f987c100a50 0x7f987c197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.556+0000 7f9881057640 1 -- 172.21.15.103:0/1312435332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c19f9b0 con 0x7f987c102600 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.557+0000 7f9881057640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c102600 0x7f987c19f310 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f987400b970 tx=0x7f987400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.557+0000 7f9879ffb640 1 -- 172.21.15.103:0/1312435332 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874019070 con 0x7f987c102600 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.557+0000 7f9879ffb640 1 -- 172.21.15.103:0/1312435332 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9874014440 con 0x7f987c102600 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.557+0000 7f9882aba640 1 -- 172.21.15.103:0/1312435332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c19fca0 con 0x7f987c102600 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.557+0000 7f9879ffb640 1 -- 172.21.15.103:0/1312435332 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98740075d0 con 0x7f987c102600 2024-09-05T02:03:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.557+0000 7f9882aba640 1 -- 172.21.15.103:0/1312435332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c1a0170 con 0x7f987c102600 2024-09-05T02:03:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.560+0000 7f9879ffb640 1 -- 172.21.15.103:0/1312435332 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f987401c070 con 0x7f987c102600 2024-09-05T02:03:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.560+0000 7f9885857740 1 Processor -- start 2024-09-05T02:03:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.560+0000 7f9885857740 1 -- start start 2024-09-05T02:03:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.560+0000 7f9885857740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 0x55c5a807e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.560+0000 7f9885857740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807e6f0 0x55c5a80831b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.560+0000 7f9885857740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a8083780 0x55c5a8085c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9885857740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c5a7fdd2f0 con 0x55c5a807e6f0 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9885857740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c5a7f85270 con 0x55c5a807ddb0 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9879ffb640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f986c0a2c30 0x7f986c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9885857740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c5a7fd92b0 con 0x55c5a8083780 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9880856640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807e6f0 0x55c5a80831b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9881057640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f986c0a2c30 0x7f986c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9880856640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807e6f0 0x55c5a80831b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57902/0 (socket says 172.21.15.103:57902) 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f987bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a8083780 0x55c5a8085c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9880856640 1 -- 172.21.15.103:0/1021465771 learned_addr learned my addr 172.21.15.103:0/1021465771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f987bfff640 1 --2- 172.21.15.103:0/1021465771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 0x55c5a807e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9879ffb640 1 -- 172.21.15.103:0/1312435332 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f98740cea50 con 0x7f987c102600 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9880856640 1 -- 172.21.15.103:0/1021465771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a8083780 msgr2=0x55c5a8085c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9880856640 1 --2- 172.21.15.103:0/1021465771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a8083780 0x55c5a8085c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9880856640 1 -- 172.21.15.103:0/1021465771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 msgr2=0x55c5a807e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9880856640 1 --2- 172.21.15.103:0/1021465771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 0x55c5a807e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9880856640 1 -- 172.21.15.103:0/1021465771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a7f84340 con 0x55c5a807e6f0 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9881057640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f986c0a2c30 0x7f986c0a50f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9874016000 tx=0x7f9874014c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9880856640 1 --2- 172.21.15.103:0/1021465771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807e6f0 0x55c5a80831b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9870009900 tx=0x7f9870031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcd6df5e8a76bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.561+0000 7f9866ffd640 1 -- 172.21.15.103:0/1021465771 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870035030 con 0x55c5a807e6f0 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9866ffd640 1 -- 172.21.15.103:0/1021465771 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9870040070 con 0x55c5a807e6f0 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9866ffd640 1 -- 172.21.15.103:0/1021465771 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870003be0 con 0x55c5a807e6f0 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9885857740 1 -- 172.21.15.103:0/1021465771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807e6f0 msgr2=0x55c5a80831b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9885857740 1 --2- 172.21.15.103:0/1021465771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807e6f0 0x55c5a80831b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9870009900 tx=0x7f9870031dc0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9885857740 1 -- 172.21.15.103:0/1021465771 shutdown_connections 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9885857740 1 --2- 172.21.15.103:0/1021465771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a8083780 0x55c5a8085c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9885857740 1 --2- 172.21.15.103:0/1021465771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807e6f0 0x55c5a80831b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9885857740 1 --2- 172.21.15.103:0/1021465771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 0x55c5a807e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9885857740 1 -- 172.21.15.103:0/1021465771 >> 172.21.15.103:0/1021465771 conn(0x55c5a8075010 msgr2=0x55c5a8075410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9885857740 1 -- 172.21.15.103:0/1021465771 shutdown_connections 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.562+0000 7f9885857740 1 -- 172.21.15.103:0/1021465771 wait complete. 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9885857740 1 -- start start 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9885857740 1 Processor -- start 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9885857740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8087fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9885857740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 0x55c5a80884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9885857740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a8083780 0x55c5a808f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9885857740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c5a80866f0 con 0x55c5a807ddb0 2024-09-05T02:03:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9885857740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c5a7f66290 con 0x55c5a8083780 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9885857740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8086870 con 0x55c5a807e6f0 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9885857740 10 client.0 _ll_register_callbacks cb 0x55c5a7f97140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f987bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8087fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f987bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8087fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57908/0 (socket says 172.21.15.103:57908) 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f987bfff640 1 -- 172.21.15.103:0/3738419136 learned_addr learned my addr 172.21.15.103:0/3738419136 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f987bfff640 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a8083780 0x55c5a808f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f9880856640 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 0x55c5a80884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f987bfff640 1 -- 172.21.15.103:0/3738419136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 msgr2=0x55c5a80884f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f987bfff640 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 0x55c5a80884f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f987bfff640 1 -- 172.21.15.103:0/3738419136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a8083780 msgr2=0x55c5a808f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f987bfff640 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a8083780 0x55c5a808f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.563+0000 7f987bfff640 1 -- 172.21.15.103:0/3738419136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a7cf0210 con 0x55c5a807ddb0 2024-09-05T02:03:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f987bfff640 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8087fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f986800c640 tx=0x7f98680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6258fefbf8871e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f985bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f985bfff640 1 -- 172.21.15.103:0/3738419136 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868011e90 con 0x55c5a807ddb0 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f985bfff640 1 -- 172.21.15.103:0/3738419136 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9868004030 con 0x55c5a807ddb0 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f9885857740 1 -- 172.21.15.103:0/3738419136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9870040070 con 0x55c5a807ddb0 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f985bfff640 1 -- 172.21.15.103:0/3738419136 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9868026050 con 0x55c5a807ddb0 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f9885857740 10 client.15489 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f9885857740 1 -- 172.21.15.103:0/3738419136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5a7dbc620 con 0x55c5a807ddb0 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f985bfff640 1 -- 172.21.15.103:0/3738419136 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9870040070 con 0x55c5a807ddb0 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f985bfff640 1 -- 172.21.15.103:0/3738419136 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9868003cd0 con 0x55c5a807ddb0 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f9885857740 10 client.15489 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f9885857740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f9885857740 20 client.15489 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f9885857740 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55c5a8093cd0 0x55c5a8096150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f9885857740 4 client.15489 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.564+0000 7f9885857740 1 -- 172.21.15.103:0/3738419136 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5a7feaa20 con 0x55c5a8093cd0 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.565+0000 7f9880856640 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55c5a8093cd0 0x55c5a8096150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.565+0000 7f985bfff640 10 client.15489 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.565+0000 7f9880856640 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55c5a8093cd0 0x55c5a8096150 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c53999c5b0d7 server_cookie=68fe95051fda5663 in_seq=0 out_seq=0 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f985bfff640 1 -- 172.21.15.103:0/3738419136 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5a7feaa20 con 0x55c5a8093cd0 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f985bfff640 10 client.15489 handle_command_reply: tid=0 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 client.15489 shutdown 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 -- 172.21.15.103:0/3738419136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 msgr2=0x55c5a8087fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8087fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f986800c640 tx=0x7f98680046c0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 -- 172.21.15.103:0/3738419136 shutdown_connections 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55c5a8093cd0 0x55c5a8096150 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a8083780 0x55c5a808f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 0x55c5a80884f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 --2- 172.21.15.103:0/3738419136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8087fb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f985bfff640 0 client.15489 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 -- 172.21.15.103:0/3738419136 >> 172.21.15.103:0/3738419136 conn(0x55c5a8075010 msgr2=0x55c5a8076820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 -- 172.21.15.103:0/3738419136 shutdown_connections 2024-09-05T02:03:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 1 -- 172.21.15.103:0/3738419136 wait complete. 2024-09-05T02:03:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.566+0000 7f9885857740 20 client.15489 trim_cache size 0 max 16384 2024-09-05T02:03:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.580+0000 7f9885857740 1 Processor -- start 2024-09-05T02:03:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.580+0000 7f9885857740 1 -- start start 2024-09-05T02:03:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f9885857740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8088580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f9885857740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807e6f0 0x55c5a808f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f9885857740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a8083780 0x55c5a80740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f9885857740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a84a0 con 0x55c5a807ddb0 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f9885857740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a9fe0 con 0x55c5a807e6f0 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f9885857740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a8b70 con 0x55c5a8083780 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f987bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8088580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f987bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8088580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57920/0 (socket says 172.21.15.103:57920) 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f987bfff640 1 -- 172.21.15.103:0/3056466035 learned_addr learned my addr 172.21.15.103:0/3056466035 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f987bfff640 1 -- 172.21.15.103:0/3056466035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a8083780 msgr2=0x55c5a80740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f987bfff640 1 --2- 172.21.15.103:0/3056466035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a8083780 0x55c5a80740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f987bfff640 1 -- 172.21.15.103:0/3056466035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807e6f0 msgr2=0x55c5a808f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f987bfff640 1 --2- 172.21.15.103:0/3056466035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807e6f0 0x55c5a808f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f987bfff640 1 -- 172.21.15.103:0/3056466035 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a8087af0 con 0x55c5a807ddb0 2024-09-05T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f987bfff640 1 --2- 172.21.15.103:0/3056466035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8088580 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f986800ce40 tx=0x7f986800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c26345b6fd8c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f98667fc640 1 -- 172.21.15.103:0/3056466035 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986802f070 con 0x55c5a807ddb0 2024-09-05T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f98667fc640 1 -- 172.21.15.103:0/3056466035 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9868011e90 con 0x55c5a807ddb0 2024-09-05T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.581+0000 7f98667fc640 1 -- 172.21.15.103:0/3056466035 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98680216f0 con 0x55c5a807ddb0 2024-09-05T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 -- 172.21.15.103:0/3056466035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 msgr2=0x55c5a8088580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 --2- 172.21.15.103:0/3056466035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8088580 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f986800ce40 tx=0x7f986800c8d0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 -- 172.21.15.103:0/3056466035 shutdown_connections 2024-09-05T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 --2- 172.21.15.103:0/3056466035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a8083780 0x55c5a80740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 --2- 172.21.15.103:0/3056466035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807e6f0 0x55c5a808f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 --2- 172.21.15.103:0/3056466035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a807ddb0 0x55c5a8088580 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 -- 172.21.15.103:0/3056466035 >> 172.21.15.103:0/3056466035 conn(0x55c5a8075010 msgr2=0x55c5a80739a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 -- 172.21.15.103:0/3056466035 shutdown_connections 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 -- 172.21.15.103:0/3056466035 wait complete. 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 -- start start 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.582+0000 7f9885857740 1 Processor -- start 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f9885857740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 0x55c5a808d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f9885857740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 0x55c5a808d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f9885857740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a8083780 0x55c5a808e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f9885857740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a9910 con 0x55c5a8083780 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f9885857740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a9240 con 0x55c5a807ddb0 2024-09-05T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f9885857740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a6480 con 0x55c5a807e6f0 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f9885857740 10 client.0 _ll_register_callbacks cb 0x7f986800cd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a8083780 0x55c5a808e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a8083780 0x55c5a808e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57936/0 (socket says 172.21.15.103:57936) 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 -- 172.21.15.103:0/4053449986 learned_addr learned my addr 172.21.15.103:0/4053449986 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 0x55c5a808d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f9880856640 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 0x55c5a808d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 -- 172.21.15.103:0/4053449986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 msgr2=0x55c5a808d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 0x55c5a808d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 -- 172.21.15.103:0/4053449986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 msgr2=0x55c5a808d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 0x55c5a808d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 -- 172.21.15.103:0/4053449986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a8086930 con 0x55c5a8083780 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f9880856640 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 0x55c5a808d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f987bfff640 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a8083780 0x55c5a808e7d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9868015af0 tx=0x7f986801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff67c305ecce49da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.583+0000 7f985bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f985bfff640 1 -- 172.21.15.103:0/4053449986 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986802f040 con 0x55c5a8083780 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f985bfff640 1 -- 172.21.15.103:0/4053449986 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9868012ce0 con 0x55c5a8083780 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9885857740 1 -- 172.21.15.103:0/4053449986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9868011e90 con 0x55c5a8083780 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f985bfff640 1 -- 172.21.15.103:0/4053449986 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9868005030 con 0x55c5a8083780 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9885857740 10 client.15495 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9885857740 1 -- 172.21.15.103:0/4053449986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5a80fefd0 con 0x55c5a8083780 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f985bfff640 1 -- 172.21.15.103:0/4053449986 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9868011e90 con 0x55c5a8083780 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f985bfff640 1 -- 172.21.15.103:0/4053449986 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9868004b40 con 0x55c5a8083780 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9885857740 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9885857740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9885857740 20 client.15495 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9885857740 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55c5a80ff7f0 0x55c5a8101c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9885857740 4 client.15495 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9885857740 1 -- 172.21.15.103:0/4053449986 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5a7fe4d70 con 0x55c5a80ff7f0 2024-09-05T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.584+0000 7f9880856640 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55c5a80ff7f0 0x55c5a8101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.585+0000 7f985bfff640 10 client.15495 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.585+0000 7f9880856640 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55c5a80ff7f0 0x55c5a8101c10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3cd221871e40a7 server_cookie=b39f76dd85ee79f7 in_seq=0 out_seq=0 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f985bfff640 1 -- 172.21.15.103:0/4053449986 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c5a7fe4d70 con 0x55c5a80ff7f0 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f985bfff640 10 client.15495 handle_command_reply: tid=0 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 client.15495 shutdown 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 -- 172.21.15.103:0/4053449986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a8083780 msgr2=0x55c5a808e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a8083780 0x55c5a808e7d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9868015af0 tx=0x7f986801d040 comp rx=0 tx=0).stop 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 -- 172.21.15.103:0/4053449986 shutdown_connections 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55c5a80ff7f0 0x55c5a8101c10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5a8083780 0x55c5a808e7d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5a807e6f0 0x55c5a808d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f985bfff640 0 client.15495 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 --2- 172.21.15.103:0/4053449986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c5a807ddb0 0x55c5a808d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 -- 172.21.15.103:0/4053449986 >> 172.21.15.103:0/4053449986 conn(0x55c5a8075010 msgr2=0x55c5a80741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 -- 172.21.15.103:0/4053449986 shutdown_connections 2024-09-05T02:03:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 1 -- 172.21.15.103:0/4053449986 wait complete. 2024-09-05T02:03:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.593+0000 7f9885857740 20 client.15495 trim_cache size 0 max 16384 2024-09-05T02:03:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:03:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:03:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:03:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:03:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 -- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f986c0a2c30 msgr2=0x7f986c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f986c0a2c30 0x7f986c0a50f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9874016000 tx=0x7f9874014c00 comp rx=0 tx=0).stop 2024-09-05T02:03:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 -- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c102600 msgr2=0x7f987c19f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c102600 0x7f987c19f310 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f987400b970 tx=0x7f987400be30 comp rx=0 tx=0).stop 2024-09-05T02:03:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 -- 172.21.15.103:0/1312435332 shutdown_connections 2024-09-05T02:03:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f987c102600 0x7f987c19f310 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f986c0a2c30 0x7f986c0a50f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f987c101c40 0x7f987c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 --2- 172.21.15.103:0/1312435332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f987c100a50 0x7f987c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 -- 172.21.15.103:0/1312435332 >> 172.21.15.103:0/1312435332 conn(0x7f987c0fc1b0 msgr2=0x7f987c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 -- 172.21.15.103:0/1312435332 shutdown_connections 2024-09-05T02:03:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:07.594+0000 7f9882aba640 1 -- 172.21.15.103:0/1312435332 wait complete. 2024-09-05T02:03:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:08 smithi167 ceph-mon[35122]: pgmap v371: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-05T02:03:08.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:08 smithi139 ceph-mon[34262]: pgmap v371: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-05T02:03:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:08 smithi103 ceph-mon[30902]: pgmap v371: 129 pgs: 129 active+clean; 670 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-05T02:03:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:10 smithi167 ceph-mon[35122]: pgmap v372: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-05T02:03:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:10 smithi139 ceph-mon[34262]: pgmap v372: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-05T02:03:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:10 smithi103 ceph-mon[30902]: pgmap v372: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-05T02:03:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:12 smithi167 ceph-mon[35122]: pgmap v373: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-05T02:03:12.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:12 smithi139 ceph-mon[34262]: pgmap v373: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-05T02:03:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:12 smithi103 ceph-mon[30902]: pgmap v373: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-05T02:03:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:14 smithi167 ceph-mon[35122]: pgmap v374: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-05T02:03:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:14 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:03:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:14 smithi139 ceph-mon[34262]: pgmap v374: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-05T02:03:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:14 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:03:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:14 smithi103 ceph-mon[30902]: pgmap v374: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-05T02:03:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:14 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:03:15.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:03:15.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:03:15.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:03:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:03:15.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:03:15.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:03:15.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:03:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:03:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:03:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:16 smithi167 ceph-mon[35122]: pgmap v375: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-09-05T02:03:16.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:16 smithi139 ceph-mon[34262]: pgmap v375: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-09-05T02:03:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:16 smithi103 ceph-mon[30902]: pgmap v375: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-09-05T02:03:17.761 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.2 2024-09-05T02:03:17.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:03:18.025 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:03:18.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:18 smithi103 ceph-mon[30902]: pgmap v376: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-05T02:03:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:18 smithi167 ceph-mon[35122]: pgmap v376: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-05T02:03:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:18 smithi139 ceph-mon[34262]: pgmap v376: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-05T02:03:20.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.981+0000 7ffbc6726640 1 -- 172.21.15.103:0/202223844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc010c2f0 msgr2=0x7ffbc010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.981+0000 7ffbc6726640 1 --2- 172.21.15.103:0/202223844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc010c2f0 0x7ffbc010e790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009ca0 tx=0x7ffbb802a8c0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.981+0000 7ffbc6726640 1 -- 172.21.15.103:0/202223844 shutdown_connections 2024-09-05T02:03:20.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.981+0000 7ffbc6726640 1 --2- 172.21.15.103:0/202223844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc010c2f0 0x7ffbc010e790 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.981+0000 7ffbc6726640 1 --2- 172.21.15.103:0/202223844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc01037a0 0x7ffbc010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.981+0000 7ffbc6726640 1 --2- 172.21.15.103:0/202223844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc0102df0 0x7ffbc01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.981+0000 7ffbc6726640 1 -- 172.21.15.103:0/202223844 >> 172.21.15.103:0/202223844 conn(0x7ffbc00fc7d0 msgr2=0x7ffbc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.981+0000 7ffbc6726640 1 -- 172.21.15.103:0/202223844 shutdown_connections 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.981+0000 7ffbc6726640 1 -- 172.21.15.103:0/202223844 wait complete. 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.982+0000 7ffbc6726640 1 Processor -- start 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.982+0000 7ffbc6726640 1 -- start start 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.982+0000 7ffbc6726640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc0102df0 0x7ffbc01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.982+0000 7ffbc6726640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc01037a0 0x7ffbc01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.982+0000 7ffbc6726640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc010c2f0 0x7ffbc019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.982+0000 7ffbc6726640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0111fa0 con 0x7ffbc01037a0 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.982+0000 7ffbc6726640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0111e20 con 0x7ffbc010c2f0 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.982+0000 7ffbc6726640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0112120 con 0x7ffbc0102df0 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc01037a0 0x7ffbc01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc01037a0 0x7ffbc01989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45886/0 (socket says 172.21.15.103:45886) 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbf7fe640 1 -- 172.21.15.103:0/2148965931 learned_addr learned my addr 172.21.15.103:0/2148965931 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbc4cc3640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc010c2f0 0x7ffbc019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbffff640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc0102df0 0x7ffbc01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbf7fe640 1 -- 172.21.15.103:0/2148965931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc0102df0 msgr2=0x7ffbc01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbf7fe640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc0102df0 0x7ffbc01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbf7fe640 1 -- 172.21.15.103:0/2148965931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc010c2f0 msgr2=0x7ffbc019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbf7fe640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc010c2f0 0x7ffbc019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbf7fe640 1 -- 172.21.15.103:0/2148965931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc019d460 con 0x7ffbc01037a0 2024-09-05T02:03:20.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbf7fe640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc01037a0 0x7ffbc01989f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c8c0 tx=0x7ffbb400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbd7fa640 1 -- 172.21.15.103:0/2148965931 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4007bb0 con 0x7ffbc01037a0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc019d750 con 0x7ffbc01037a0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbd7fa640 1 -- 172.21.15.103:0/2148965931 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbb4007d50 con 0x7ffbc01037a0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbbd7fa640 1 -- 172.21.15.103:0/2148965931 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40187c0 con 0x7ffbc01037a0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.983+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc01a5030 con 0x7ffbc01037a0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.984+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc01031d0 con 0x7ffbc01037a0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.985+0000 7ffbbd7fa640 1 -- 172.21.15.103:0/2148965931 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffbb400f450 con 0x7ffbc01037a0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.986+0000 7ffbbd7fa640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffba80a2b80 0x7ffba80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.986+0000 7ffbbffff640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffba80a2b80 0x7ffba80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.986+0000 7ffbbd7fa640 1 -- 172.21.15.103:0/2148965931 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffbb40cf450 con 0x7ffbc01037a0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.987+0000 7ffbbffff640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffba80a2b80 0x7ffba80a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005fd0 tx=0x7ffbb0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:20.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:19.991+0000 7ffbbd7fa640 1 -- 172.21.15.103:0/2148965931 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbb4014030 con 0x7ffbc01037a0 2024-09-05T02:03:20.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.211+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbc01072a0 con 0x7ffbc01037a0 2024-09-05T02:03:20.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.212+0000 7ffbbd7fa640 1 -- 172.21.15.103:0/2148965931 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7ffbb40afe20 con 0x7ffbc01037a0 2024-09-05T02:03:20.213 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:03:20.213 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":29,"btime":"2024-09-05T02:02:41:223802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:02:41.148036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:03:20.214 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 29 2024-09-05T02:03:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffba80a2b80 msgr2=0x7ffba80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffba80a2b80 0x7ffba80a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005fd0 tx=0x7ffbb0005850 comp rx=0 tx=0).stop 2024-09-05T02:03:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc01037a0 msgr2=0x7ffbc01989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc01037a0 0x7ffbc01989f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c8c0 tx=0x7ffbb400cd80 comp rx=0 tx=0).stop 2024-09-05T02:03:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 shutdown_connections 2024-09-05T02:03:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc010c2f0 0x7ffbc019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc01037a0 0x7ffbc01989f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffba80a2b80 0x7ffba80a5040 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 --2- 172.21.15.103:0/2148965931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc0102df0 0x7ffbc01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 >> 172.21.15.103:0/2148965931 conn(0x7ffbc00fc7d0 msgr2=0x7ffbc00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 shutdown_connections 2024-09-05T02:03:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:20.214+0000 7ffbc6726640 1 -- 172.21.15.103:0/2148965931 wait complete. 2024-09-05T02:03:20.586 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:20 smithi103 ceph-mon[30902]: pgmap v377: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-05T02:03:20.586 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:20 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2148965931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:03:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:20 smithi167 ceph-mon[35122]: pgmap v377: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-05T02:03:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:20 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2148965931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:03:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:20 smithi139 ceph-mon[34262]: pgmap v377: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-05T02:03:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:20 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2148965931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:03:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.684+0000 7f1b60277640 1 -- 172.21.15.103:0/3643438929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b58103170 msgr2=0x7f1b58103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.684+0000 7f1b60277640 1 --2- 172.21.15.103:0/3643438929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b58103170 0x7f1b58103570 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009900 tx=0x7f1b5402a410 comp rx=0 tx=0).stop 2024-09-05T02:03:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.684+0000 7f1b60277640 1 -- 172.21.15.103:0/3643438929 shutdown_connections 2024-09-05T02:03:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.684+0000 7f1b60277640 1 --2- 172.21.15.103:0/3643438929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b581005a0 0x7f1b5810a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.684+0000 7f1b60277640 1 --2- 172.21.15.103:0/3643438929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b580ffbe0 0x7f1b58100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.684+0000 7f1b60277640 1 --2- 172.21.15.103:0/3643438929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b58103170 0x7f1b58103570 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.684+0000 7f1b60277640 1 -- 172.21.15.103:0/3643438929 >> 172.21.15.103:0/3643438929 conn(0x7f1b580fb930 msgr2=0x7f1b580fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.684+0000 7f1b60277640 1 -- 172.21.15.103:0/3643438929 shutdown_connections 2024-09-05T02:03:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.684+0000 7f1b60277640 1 -- 172.21.15.103:0/3643438929 wait complete. 2024-09-05T02:03:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b60277640 1 Processor -- start 2024-09-05T02:03:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b60277640 1 -- start start 2024-09-05T02:03:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b60277640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b580ffbe0 0x7f1b58199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b60277640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b581005a0 0x7f1b5819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b60277640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b58103170 0x7f1b581a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b60277640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810d690 con 0x7f1b581005a0 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b60277640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810d510 con 0x7f1b58103170 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b60277640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810d810 con 0x7f1b580ffbe0 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5d812640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b581005a0 0x7f1b5819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5d812640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b581005a0 0x7f1b5819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45898/0 (socket says 172.21.15.103:45898) 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5d812640 1 -- 172.21.15.103:0/3389063840 learned_addr learned my addr 172.21.15.103:0/3389063840 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5e013640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b580ffbe0 0x7f1b58199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5e814640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b58103170 0x7f1b581a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5d812640 1 -- 172.21.15.103:0/3389063840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b580ffbe0 msgr2=0x7f1b58199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5d812640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b580ffbe0 0x7f1b58199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5d812640 1 -- 172.21.15.103:0/3389063840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b58103170 msgr2=0x7f1b581a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5d812640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b58103170 0x7f1b581a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5d812640 1 -- 172.21.15.103:0/3389063840 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b581a1af0 con 0x7f1b581005a0 2024-09-05T02:03:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.685+0000 7f1b5d812640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b581005a0 0x7f1b5819a410 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800c920 tx=0x7f1b4800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.686+0000 7f1b4f7fe640 1 -- 172.21.15.103:0/3389063840 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48007c70 con 0x7f1b581005a0 2024-09-05T02:03:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.686+0000 7f1b4f7fe640 1 -- 172.21.15.103:0/3389063840 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b48007e10 con 0x7f1b581005a0 2024-09-05T02:03:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.686+0000 7f1b60277640 1 -- 172.21.15.103:0/3389063840 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b581a1de0 con 0x7f1b581005a0 2024-09-05T02:03:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.686+0000 7f1b4f7fe640 1 -- 172.21.15.103:0/3389063840 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48018820 con 0x7f1b581005a0 2024-09-05T02:03:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.686+0000 7f1b60277640 1 -- 172.21.15.103:0/3389063840 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b581a22a0 con 0x7f1b581005a0 2024-09-05T02:03:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b63011740 1 Processor -- start 2024-09-05T02:03:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b63011740 1 -- start start 2024-09-05T02:03:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b4f7fe640 1 -- 172.21.15.103:0/3389063840 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1b480189c0 con 0x7f1b581005a0 2024-09-05T02:03:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b63011740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 0x56491a01eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b63011740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01f2e0 0x56491a023da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b63011740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a024370 0x56491a026830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b63011740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564919f822f0 con 0x56491a01f2e0 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b63011740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564919f2a270 con 0x56491a01e9a0 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b63011740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564919f7e2b0 con 0x56491a024370 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b5e013640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01f2e0 0x56491a023da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b5e013640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01f2e0 0x56491a023da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45904/0 (socket says 172.21.15.103:45904) 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.689+0000 7f1b5e013640 1 -- 172.21.15.103:0/2848106119 learned_addr learned my addr 172.21.15.103:0/2848106119 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b5e814640 1 --2- 172.21.15.103:0/2848106119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 0x56491a01eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b5d812640 1 --2- 172.21.15.103:0/2848106119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a024370 0x56491a026830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b5e013640 1 -- 172.21.15.103:0/2848106119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a024370 msgr2=0x56491a026830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b5e013640 1 --2- 172.21.15.103:0/2848106119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a024370 0x56491a026830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b5e013640 1 -- 172.21.15.103:0/2848106119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 msgr2=0x56491a01eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b5e013640 1 --2- 172.21.15.103:0/2848106119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 0x56491a01eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b5e013640 1 -- 172.21.15.103:0/2848106119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564919f29340 con 0x56491a01f2e0 2024-09-05T02:03:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b5e013640 1 --2- 172.21.15.103:0/2848106119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01f2e0 0x56491a023da0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1b540314e0 tx=0x7f1b54032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569f7e3627b1beb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b4f7fe640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b440a2c30 0x7f1b440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b4cbfe640 1 -- 172.21.15.103:0/2848106119 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5403f070 con 0x56491a01f2e0 2024-09-05T02:03:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b4f7fe640 1 -- 172.21.15.103:0/3389063840 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1b480cf5f0 con 0x7f1b581005a0 2024-09-05T02:03:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.690+0000 7f1b5e814640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b440a2c30 0x7f1b440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b4cbfe640 1 -- 172.21.15.103:0/2848106119 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b54032e50 con 0x56491a01f2e0 2024-09-05T02:03:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b4cbfe640 1 -- 172.21.15.103:0/2848106119 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54035070 con 0x56491a01f2e0 2024-09-05T02:03:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b5e814640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b440a2c30 0x7f1b440a50f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000be30 tx=0x7f1b5000b580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b63011740 1 -- 172.21.15.103:0/2848106119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01f2e0 msgr2=0x56491a023da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b63011740 1 --2- 172.21.15.103:0/2848106119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01f2e0 0x56491a023da0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1b540314e0 tx=0x7f1b54032c60 comp rx=0 tx=0).stop 2024-09-05T02:03:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b63011740 1 -- 172.21.15.103:0/2848106119 shutdown_connections 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b63011740 1 --2- 172.21.15.103:0/2848106119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a024370 0x56491a026830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b63011740 1 --2- 172.21.15.103:0/2848106119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01f2e0 0x56491a023da0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b63011740 1 --2- 172.21.15.103:0/2848106119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 0x56491a01eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b63011740 1 -- 172.21.15.103:0/2848106119 >> 172.21.15.103:0/2848106119 conn(0x56491a01a010 msgr2=0x56491a01a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b63011740 1 -- 172.21.15.103:0/2848106119 shutdown_connections 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.691+0000 7f1b63011740 1 -- 172.21.15.103:0/2848106119 wait complete. 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b63011740 1 -- start start 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b63011740 1 Processor -- start 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b63011740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01e9a0 0x56491a028b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b63011740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01f2e0 0x56491a0290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b63011740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a0300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b63011740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56491a0272e0 con 0x56491a024370 2024-09-05T02:03:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b63011740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564919f0b290 con 0x56491a01f2e0 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b63011740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56491a027460 con 0x56491a01e9a0 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b63011740 10 client.0 _ll_register_callbacks cb 0x564919f3c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b5e814640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a0300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b5d812640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01f2e0 0x56491a0290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b5e814640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a0300a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45916/0 (socket says 172.21.15.103:45916) 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b5e814640 1 -- 172.21.15.103:0/3013785842 learned_addr learned my addr 172.21.15.103:0/3013785842 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.692+0000 7f1b5e013640 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01e9a0 0x56491a028b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b5e814640 1 -- 172.21.15.103:0/3013785842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01e9a0 msgr2=0x56491a028b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b5e814640 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01e9a0 0x56491a028b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b5e814640 1 -- 172.21.15.103:0/3013785842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01f2e0 msgr2=0x56491a0290d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b5e814640 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01f2e0 0x56491a0290d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b5e814640 1 -- 172.21.15.103:0/3013785842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564919c95210 con 0x56491a024370 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b5d812640 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01f2e0 0x56491a0290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b5e814640 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a0300a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000eb90 tx=0x7f1b50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de83fd2ddcb92e08 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b41ffb640 1 -- 172.21.15.103:0/3013785842 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50025400 con 0x56491a024370 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b41ffb640 1 -- 172.21.15.103:0/3013785842 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1b5001d400 con 0x56491a024370 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b63011740 1 -- 172.21.15.103:0/3013785842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b54032e50 con 0x56491a024370 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b41ffb640 1 -- 172.21.15.103:0/3013785842 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b5001d600 con 0x56491a024370 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b63011740 10 client.15513 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b63011740 1 -- 172.21.15.103:0/3013785842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564919d61620 con 0x56491a024370 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.693+0000 7f1b41ffb640 1 -- 172.21.15.103:0/3013785842 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b54032e50 con 0x56491a024370 2024-09-05T02:03:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b41ffb640 1 -- 172.21.15.103:0/3013785842 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1b50025870 con 0x56491a024370 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b63011740 10 client.15513 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b63011740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b63011740 20 client.15513 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b63011740 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56491a034810 0x56491a036c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b63011740 4 client.15513 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b63011740 1 -- 172.21.15.103:0/3013785842 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564919f8fa20 con 0x56491a034810 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b5e013640 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56491a034810 0x56491a036c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b41ffb640 10 client.15513 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.694+0000 7f1b5e013640 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56491a034810 0x56491a036c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d75f8ebe27ce3b server_cookie=38f720d8ed3f4775 in_seq=0 out_seq=0 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b41ffb640 1 -- 172.21.15.103:0/3013785842 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564919f8fa20 con 0x56491a034810 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b41ffb640 10 client.15513 handle_command_reply: tid=0 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 client.15513 shutdown 2024-09-05T02:03:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 -- 172.21.15.103:0/3013785842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 msgr2=0x56491a0300a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a0300a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000eb90 tx=0x7f1b50004a40 comp rx=0 tx=0).stop 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 -- 172.21.15.103:0/3013785842 shutdown_connections 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56491a034810 0x56491a036c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a0300a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01f2e0 0x56491a0290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 --2- 172.21.15.103:0/3013785842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01e9a0 0x56491a028b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b41ffb640 0 client.15513 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 -- 172.21.15.103:0/3013785842 >> 172.21.15.103:0/3013785842 conn(0x56491a01a010 msgr2=0x56491a01b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 -- 172.21.15.103:0/3013785842 shutdown_connections 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.695+0000 7f1b63011740 1 -- 172.21.15.103:0/3013785842 wait complete. 2024-09-05T02:03:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.696+0000 7f1b63011740 20 client.15513 trim_cache size 0 max 16384 2024-09-05T02:03:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b63011740 1 Processor -- start 2024-09-05T02:03:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b63011740 1 -- start start 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b63011740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 0x56491a0190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b63011740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 0x56491a030060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b63011740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a03b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b63011740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581ac7f0 con 0x56491a024370 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b63011740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581ae330 con 0x56491a01e9a0 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b63011740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581acec0 con 0x56491a01f2e0 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b5e814640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a03b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b5e814640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a03b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45920/0 (socket says 172.21.15.103:45920) 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.724+0000 7f1b5e814640 1 -- 172.21.15.103:0/3993419702 learned_addr learned my addr 172.21.15.103:0/3993419702 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b5e013640 1 --2- 172.21.15.103:0/3993419702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 0x56491a0190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b5d812640 1 --2- 172.21.15.103:0/3993419702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 0x56491a030060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b5e814640 1 -- 172.21.15.103:0/3993419702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 msgr2=0x56491a030060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b5e814640 1 --2- 172.21.15.103:0/3993419702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 0x56491a030060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b5e814640 1 -- 172.21.15.103:0/3993419702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 msgr2=0x56491a0190b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b5e814640 1 --2- 172.21.15.103:0/3993419702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 0x56491a0190b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b5e814640 1 -- 172.21.15.103:0/3993419702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56491a019e30 con 0x56491a024370 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b5e814640 1 --2- 172.21.15.103:0/3993419702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a03b590 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1b50010e80 tx=0x7f1b5000b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a607bc85f56dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b43fff640 1 -- 172.21.15.103:0/3993419702 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50037070 con 0x56491a024370 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b43fff640 1 -- 172.21.15.103:0/3993419702 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b5000e750 con 0x56491a024370 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.725+0000 7f1b43fff640 1 -- 172.21.15.103:0/3993419702 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50025aa0 con 0x56491a024370 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 -- 172.21.15.103:0/3993419702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 msgr2=0x56491a03b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 --2- 172.21.15.103:0/3993419702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a03b590 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1b50010e80 tx=0x7f1b5000b5f0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 -- 172.21.15.103:0/3993419702 shutdown_connections 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 --2- 172.21.15.103:0/3993419702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a024370 0x56491a03b590 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 --2- 172.21.15.103:0/3993419702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 0x56491a030060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 --2- 172.21.15.103:0/3993419702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a01e9a0 0x56491a0190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 -- 172.21.15.103:0/3993419702 >> 172.21.15.103:0/3993419702 conn(0x56491a01a010 msgr2=0x56491a0189a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 -- 172.21.15.103:0/3993419702 shutdown_connections 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 -- 172.21.15.103:0/3993419702 wait complete. 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 -- start start 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.726+0000 7f1b63011740 1 Processor -- start 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b63011740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01e9a0 0x56491a02d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b63011740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 0x56491a02da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b63011740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a024370 0x56491a02ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b63011740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581adc60 con 0x56491a01e9a0 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b63011740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581ad590 con 0x56491a024370 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b63011740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b581aa7d0 con 0x56491a01f2e0 2024-09-05T02:03:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b63011740 10 client.0 _ll_register_callbacks cb 0x7f1b5000ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e013640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01e9a0 0x56491a02d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e013640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01e9a0 0x56491a02d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45928/0 (socket says 172.21.15.103:45928) 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e013640 1 -- 172.21.15.103:0/1854494616 learned_addr learned my addr 172.21.15.103:0/1854494616 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e814640 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a024370 0x56491a02ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e013640 1 -- 172.21.15.103:0/1854494616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 msgr2=0x56491a02da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5d812640 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 0x56491a02da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e013640 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 0x56491a02da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e013640 1 -- 172.21.15.103:0/1854494616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a024370 msgr2=0x56491a02ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e013640 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a024370 0x56491a02ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e013640 1 -- 172.21.15.103:0/1854494616 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56491a02f0a0 con 0x56491a01e9a0 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e814640 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a024370 0x56491a02ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5d812640 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 0x56491a02da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:03:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b5e013640 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01e9a0 0x56491a02d540 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1b5402a930 tx=0x7f1b54009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e67728239b450c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.727+0000 7f1b41ffb640 1 -- 172.21.15.103:0/1854494616 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54035040 con 0x56491a01e9a0 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b41ffb640 1 -- 172.21.15.103:0/1854494616 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1b54004030 con 0x56491a01e9a0 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b63011740 1 -- 172.21.15.103:0/1854494616 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564919f89d70 con 0x56491a01e9a0 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b41ffb640 1 -- 172.21.15.103:0/1854494616 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b5405e050 con 0x56491a01e9a0 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b63011740 10 client.15519 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b63011740 1 -- 172.21.15.103:0/1854494616 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56491a0303b0 con 0x56491a01e9a0 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b41ffb640 1 -- 172.21.15.103:0/1854494616 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564919f89d70 con 0x56491a01e9a0 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b41ffb640 1 -- 172.21.15.103:0/1854494616 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1b5403b460 con 0x56491a01e9a0 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b63011740 10 client.15519 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b63011740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b63011740 20 client.15519 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b63011740 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56491a09f930 0x56491a0a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b63011740 4 client.15519 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:03:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b63011740 1 -- 172.21.15.103:0/1854494616 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564919f06900 con 0x56491a09f930 2024-09-05T02:03:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.728+0000 7f1b5d812640 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56491a09f930 0x56491a0a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.729+0000 7f1b5d812640 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56491a09f930 0x56491a0a1d50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b668d38a5e9bdb5 server_cookie=a60dee68a1091b3a in_seq=0 out_seq=0 2024-09-05T02:03:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.729+0000 7f1b41ffb640 10 client.15519 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.734+0000 7f1b41ffb640 1 -- 172.21.15.103:0/1854494616 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564919f06900 con 0x56491a09f930 2024-09-05T02:03:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.734+0000 7f1b41ffb640 10 client.15519 handle_command_reply: tid=0 2024-09-05T02:03:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.734+0000 7f1b63011740 1 client.15519 shutdown 2024-09-05T02:03:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 -- 172.21.15.103:0/1854494616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01e9a0 msgr2=0x56491a02d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01e9a0 0x56491a02d540 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1b5402a930 tx=0x7f1b54009260 comp rx=0 tx=0).stop 2024-09-05T02:03:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 -- 172.21.15.103:0/1854494616 shutdown_connections 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56491a09f930 0x56491a0a1d50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56491a024370 0x56491a02ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56491a01f2e0 0x56491a02da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 --2- 172.21.15.103:0/1854494616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56491a01e9a0 0x56491a02d540 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 -- 172.21.15.103:0/1854494616 >> 172.21.15.103:0/1854494616 conn(0x56491a01a010 msgr2=0x56491a0189a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b41ffb640 0 client.15519 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 -- 172.21.15.103:0/1854494616 shutdown_connections 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 1 -- 172.21.15.103:0/1854494616 wait complete. 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b63011740 20 client.15519 trim_cache size 0 max 16384 2024-09-05T02:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:03:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:03:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:03:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b60277640 1 -- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b440a2c30 msgr2=0x7f1b440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b60277640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b440a2c30 0x7f1b440a50f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000be30 tx=0x7f1b5000b580 comp rx=0 tx=0).stop 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b60277640 1 -- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b581005a0 msgr2=0x7f1b5819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b60277640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b581005a0 0x7f1b5819a410 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800c920 tx=0x7f1b4800cde0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b60277640 1 -- 172.21.15.103:0/3389063840 shutdown_connections 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b60277640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b58103170 0x7f1b581a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b60277640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b581005a0 0x7f1b5819a410 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b60277640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b440a2c30 0x7f1b440a50f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.735+0000 7f1b60277640 1 --2- 172.21.15.103:0/3389063840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b580ffbe0 0x7f1b58199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.736+0000 7f1b60277640 1 -- 172.21.15.103:0/3389063840 >> 172.21.15.103:0/3389063840 conn(0x7f1b580fb930 msgr2=0x7f1b58100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.736+0000 7f1b60277640 1 -- 172.21.15.103:0/3389063840 shutdown_connections 2024-09-05T02:03:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:20.736+0000 7f1b60277640 1 -- 172.21.15.103:0/3389063840 wait complete. 2024-09-05T02:03:21.010 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 148.2 --await-for 45.0' 2024-09-05T02:03:21.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 148.2 --await-for 45.0 2024-09-05T02:03:21.282 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:03:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.804+0000 7f53f2063640 1 -- 172.21.15.103:0/3278342631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102ef0 msgr2=0x7f53ec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.804+0000 7f53f2063640 1 --2- 172.21.15.103:0/3278342631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102ef0 0x7f53ec10da40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009ca0 tx=0x7f53e402a950 comp rx=0 tx=0).stop 2024-09-05T02:03:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.804+0000 7f53f2063640 1 -- 172.21.15.103:0/3278342631 shutdown_connections 2024-09-05T02:03:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.804+0000 7f53f2063640 1 --2- 172.21.15.103:0/3278342631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102ef0 0x7f53ec10da40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.804+0000 7f53f2063640 1 --2- 172.21.15.103:0/3278342631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53ec102570 0x7f53ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.804+0000 7f53f2063640 1 --2- 172.21.15.103:0/3278342631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec100e20 0x7f53ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.804+0000 7f53f2063640 1 -- 172.21.15.103:0/3278342631 >> 172.21.15.103:0/3278342631 conn(0x7f53ec0fc7d0 msgr2=0x7f53ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.804+0000 7f53f2063640 1 -- 172.21.15.103:0/3278342631 shutdown_connections 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.804+0000 7f53f2063640 1 -- 172.21.15.103:0/3278342631 wait complete. 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.805+0000 7f53f2063640 1 Processor -- start 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.805+0000 7f53f2063640 1 -- start start 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.805+0000 7f53f2063640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec100e20 0x7f53ec197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.805+0000 7f53f2063640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102570 0x7f53ec197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.805+0000 7f53f2063640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53ec102ef0 0x7f53ec19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.805+0000 7f53f2063640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec110e50 con 0x7f53ec102570 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.805+0000 7f53f2063640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec110cd0 con 0x7f53ec100e20 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.805+0000 7f53f2063640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec110fd0 con 0x7f53ec102ef0 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.805+0000 7f53eaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102570 0x7f53ec197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102570 0x7f53ec197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45944/0 (socket says 172.21.15.103:45944) 2024-09-05T02:03:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eaffd640 1 -- 172.21.15.103:0/872768931 learned_addr learned my addr 172.21.15.103:0/872768931 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec100e20 0x7f53ec197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53ebfff640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53ec102ef0 0x7f53ec19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eaffd640 1 -- 172.21.15.103:0/872768931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53ec102ef0 msgr2=0x7f53ec19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eaffd640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53ec102ef0 0x7f53ec19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eaffd640 1 -- 172.21.15.103:0/872768931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec100e20 msgr2=0x7f53ec197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eaffd640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec100e20 0x7f53ec197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eaffd640 1 -- 172.21.15.103:0/872768931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec19c2c0 con 0x7f53ec102570 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec100e20 0x7f53ec197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53eaffd640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102570 0x7f53ec197850 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b520 tx=0x7f53e000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.806+0000 7f53e8ff9640 1 -- 172.21.15.103:0/872768931 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e001b070 con 0x7f53ec102570 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.807+0000 7f53f2063640 1 -- 172.21.15.103:0/872768931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec19c5b0 con 0x7f53ec102570 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.807+0000 7f53e8ff9640 1 -- 172.21.15.103:0/872768931 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e000ede0 con 0x7f53ec102570 2024-09-05T02:03:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.807+0000 7f53e8ff9640 1 -- 172.21.15.103:0/872768931 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0014340 con 0x7f53ec102570 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.807+0000 7f53f2063640 1 -- 172.21.15.103:0/872768931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec069f70 con 0x7f53ec102570 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53f443c740 1 Processor -- start 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53e8ff9640 1 -- 172.21.15.103:0/872768931 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f53e001e070 con 0x7f53ec102570 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53f443c740 1 -- start start 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53f443c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd45491b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53f443c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45492080 0x55dd4549c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53f443c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd4549c940 0x55dd4549ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53f443c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dd453f72a0 con 0x55dd45491760 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53f443c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dd4539eb10 con 0x55dd45492080 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53f443c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dd453f3260 con 0x55dd4549c940 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd45491b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd45491b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45956/0 (socket says 172.21.15.103:45956) 2024-09-05T02:03:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.810+0000 7f53ebfff640 1 -- 172.21.15.103:0/795052815 learned_addr learned my addr 172.21.15.103:0/795052815 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/795052815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45492080 0x55dd4549c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53ebfff640 1 -- 172.21.15.103:0/795052815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd4549c940 msgr2=0x55dd4549ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53ebfff640 1 --2- 172.21.15.103:0/795052815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd4549c940 0x55dd4549ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53ebfff640 1 -- 172.21.15.103:0/795052815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45492080 msgr2=0x55dd4549c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53ebfff640 1 --2- 172.21.15.103:0/795052815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45492080 0x55dd4549c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53ebfff640 1 -- 172.21.15.103:0/795052815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd45168b90 con 0x55dd45491760 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53e8ff9640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53d40a7010 0x7f53d40a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53e8ff9640 1 -- 172.21.15.103:0/872768931 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f53e00d0330 con 0x7f53ec102570 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53d40a7010 0x7f53d40a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53ebfff640 1 --2- 172.21.15.103:0/795052815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd45491b40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009a60 tx=0x7f53e402fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7490410882fbe5c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.811+0000 7f53d1ffb640 1 -- 172.21.15.103:0/795052815 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4034040 con 0x55dd45491760 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53d1ffb640 1 -- 172.21.15.103:0/795052815 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e402fb40 con 0x55dd45491760 2024-09-05T02:03:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53d1ffb640 1 -- 172.21.15.103:0/795052815 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40045d0 con 0x55dd45491760 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53f443c740 1 -- 172.21.15.103:0/795052815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 msgr2=0x55dd45491b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53d40a7010 0x7f53d40a94d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55dd4548eb50 tx=0x7f53dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53f443c740 1 --2- 172.21.15.103:0/795052815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd45491b40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009a60 tx=0x7f53e402fec0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53f443c740 1 -- 172.21.15.103:0/795052815 shutdown_connections 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53f443c740 1 --2- 172.21.15.103:0/795052815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd4549c940 0x55dd4549ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53f443c740 1 --2- 172.21.15.103:0/795052815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45492080 0x55dd4549c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53f443c740 1 --2- 172.21.15.103:0/795052815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd45491b40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.812+0000 7f53f443c740 1 -- 172.21.15.103:0/795052815 >> 172.21.15.103:0/795052815 conn(0x55dd4548dcf0 msgr2=0x55dd4548e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.813+0000 7f53f443c740 1 -- 172.21.15.103:0/795052815 shutdown_connections 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.813+0000 7f53f443c740 1 -- 172.21.15.103:0/795052815 wait complete. 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.813+0000 7f53f443c740 1 -- start start 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.813+0000 7f53f443c740 1 Processor -- start 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53f443c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd45499890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53f443c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45499dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53f443c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd4549a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53f443c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dd4549f510 con 0x55dd4549c940 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53f443c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dd4537f6e0 con 0x55dd45491760 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53f443c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dd4549f690 con 0x55dd45492080 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53eb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd45499890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53eaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45499dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53eb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd45499890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:42544/0 (socket says 172.21.15.103:42544) 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53eb7fe640 1 -- 172.21.15.103:0/1831395305 learned_addr learned my addr 172.21.15.103:0/1831395305 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.814+0000 7f53ebfff640 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd4549a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53f443c740 10 client.0 _ll_register_callbacks cb 0x55dd453a3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53eb7fe640 1 -- 172.21.15.103:0/1831395305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 msgr2=0x55dd45499dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45499dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53eb7fe640 1 -- 172.21.15.103:0/1831395305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 msgr2=0x55dd4549a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd4549a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53eb7fe640 1 -- 172.21.15.103:0/1831395305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd454a8a80 con 0x55dd45491760 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53ebfff640 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd4549a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53eaffd640 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45499dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd45499890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53dc010d20 tx=0x7f53dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcec64a353d834dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53caffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53caffd640 1 -- 172.21.15.103:0/1831395305 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc004220 con 0x55dd45491760 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53f443c740 1 -- 172.21.15.103:0/1831395305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e402fb40 con 0x55dd45491760 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.815+0000 7f53caffd640 1 -- 172.21.15.103:0/1831395305 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f53dc0043c0 con 0x55dd45491760 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.816+0000 7f53caffd640 1 -- 172.21.15.103:0/1831395305 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53dc031050 con 0x55dd45491760 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.816+0000 7f53f443c740 10 client.25025 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.816+0000 7f53f443c740 1 -- 172.21.15.103:0/1831395305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dd454ab460 con 0x55dd45491760 2024-09-05T02:03:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.816+0000 7f53caffd640 1 -- 172.21.15.103:0/1831395305 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e402fb40 con 0x55dd45491760 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53caffd640 1 -- 172.21.15.103:0/1831395305 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f53dc01b050 con 0x55dd45491760 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53f443c740 10 client.25025 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53f443c740 10 client.25025 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53f443c740 20 client.25025 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53f443c740 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd454acc00 0x55dd454af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53f443c740 4 client.25025 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53f443c740 1 -- 172.21.15.103:0/1831395305 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd454049d0 con 0x55dd454acc00 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53eaffd640 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd454acc00 0x55dd454af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53eaffd640 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd454acc00 0x55dd454af090 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65db276096bb68b server_cookie=fb083aea468adaeb in_seq=0 out_seq=0 2024-09-05T02:03:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.817+0000 7f53caffd640 10 client.25025 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:21.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53caffd640 1 -- 172.21.15.103:0/1831395305 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd454049d0 con 0x55dd454acc00 2024-09-05T02:03:21.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53caffd640 10 client.25025 handle_command_reply: tid=0 2024-09-05T02:03:21.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53f443c740 1 client.25025 shutdown 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53f443c740 1 -- 172.21.15.103:0/1831395305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 msgr2=0x55dd45499890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53f443c740 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd45499890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53dc010d20 tx=0x7f53dc0046c0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53f443c740 1 -- 172.21.15.103:0/1831395305 shutdown_connections 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53f443c740 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd454acc00 0x55dd454af090 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53f443c740 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd4549a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53f443c740 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45499dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53f443c740 1 --2- 172.21.15.103:0/1831395305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd45499890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.818+0000 7f53f443c740 1 -- 172.21.15.103:0/1831395305 >> 172.21.15.103:0/1831395305 conn(0x55dd4548dcf0 msgr2=0x55dd45493a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.819+0000 7f53caffd640 0 client.25025 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.819+0000 7f53f443c740 1 -- 172.21.15.103:0/1831395305 shutdown_connections 2024-09-05T02:03:21.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.819+0000 7f53f443c740 1 -- 172.21.15.103:0/1831395305 wait complete. 2024-09-05T02:03:21.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.819+0000 7f53f443c740 20 client.25025 trim_cache size 0 max 16384 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.835+0000 7f53f443c740 1 Processor -- start 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.835+0000 7f53f443c740 1 -- start start 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53f443c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd454b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53f443c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd454b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53f443c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd4549c940 0x55dd45496b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53f443c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1acb00 con 0x55dd45491760 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53f443c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1ae640 con 0x55dd4549c940 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53f443c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1ad1d0 con 0x55dd45492080 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd454b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd454b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45972/0 (socket says 172.21.15.103:45972) 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53eb7fe640 1 -- 172.21.15.103:0/2077398831 learned_addr learned my addr 172.21.15.103:0/2077398831 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53eaffd640 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd454b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53ebfff640 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd4549c940 0x55dd45496b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53eb7fe640 1 -- 172.21.15.103:0/2077398831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 msgr2=0x55dd454b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd454b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53eb7fe640 1 -- 172.21.15.103:0/2077398831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd4549c940 msgr2=0x55dd45496b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd4549c940 0x55dd45496b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53eb7fe640 1 -- 172.21.15.103:0/2077398831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd45489260 con 0x55dd45491760 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.836+0000 7f53ebfff640 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd4549c940 0x55dd45496b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.837+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd454b4970 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01e550 tx=0x7f53dc010490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b8ab52f51075fd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.837+0000 7f53d17fa640 1 -- 172.21.15.103:0/2077398831 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc005240 con 0x55dd45491760 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.837+0000 7f53d17fa640 1 -- 172.21.15.103:0/2077398831 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53dc03a070 con 0x55dd45491760 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.837+0000 7f53d17fa640 1 -- 172.21.15.103:0/2077398831 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc004e40 con 0x55dd45491760 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.838+0000 7f53f443c740 1 -- 172.21.15.103:0/2077398831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 msgr2=0x55dd454b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.838+0000 7f53f443c740 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd454b4970 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01e550 tx=0x7f53dc010490 comp rx=0 tx=0).stop 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.838+0000 7f53f443c740 1 -- 172.21.15.103:0/2077398831 shutdown_connections 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.838+0000 7f53f443c740 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd4549c940 0x55dd45496b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.838+0000 7f53f443c740 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd454b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.838+0000 7f53f443c740 1 --2- 172.21.15.103:0/2077398831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd45491760 0x55dd454b4970 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.838+0000 7f53f443c740 1 -- 172.21.15.103:0/2077398831 >> 172.21.15.103:0/2077398831 conn(0x55dd4548dcf0 msgr2=0x55dd4549b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.838+0000 7f53f443c740 1 -- 172.21.15.103:0/2077398831 shutdown_connections 2024-09-05T02:03:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.838+0000 7f53f443c740 1 -- 172.21.15.103:0/2077398831 wait complete. 2024-09-05T02:03:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.839+0000 7f53f443c740 1 -- start start 2024-09-05T02:03:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.839+0000 7f53f443c740 1 Processor -- start 2024-09-05T02:03:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.839+0000 7f53f443c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd454968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.839+0000 7f53f443c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45496e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.839+0000 7f53f443c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd45497360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.839+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd45497360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53f443c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1adf70 con 0x55dd4549c940 2024-09-05T02:03:21.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53f443c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1ad8a0 con 0x55dd45491760 2024-09-05T02:03:21.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53f443c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1aaae0 con 0x55dd45492080 2024-09-05T02:03:21.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.839+0000 7f53eb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd454968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd45497360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45976/0 (socket says 172.21.15.103:45976) 2024-09-05T02:03:21.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53ebfff640 1 -- 172.21.15.103:0/3251568855 learned_addr learned my addr 172.21.15.103:0/3251568855 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53eaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45496e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53f443c740 10 client.0 _ll_register_callbacks cb 0x7f53dc0109a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53eb7fe640 1 -- 172.21.15.103:0/3251568855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 msgr2=0x55dd45496e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45496e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53eb7fe640 1 -- 172.21.15.103:0/3251568855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 msgr2=0x55dd45497360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd45497360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53eb7fe640 1 -- 172.21.15.103:0/3251568855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd4548eda0 con 0x55dd45491760 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53eaffd640 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45496e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53ebfff640 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd45497360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53eb7fe640 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd454968e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01e550 tx=0x7f53dc01fcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a48cec93ae7f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53caffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.840+0000 7f53caffd640 1 -- 172.21.15.103:0/3251568855 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc028640 con 0x55dd45491760 2024-09-05T02:03:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53f443c740 1 -- 172.21.15.103:0/3251568855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53dc03a070 con 0x55dd45491760 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53caffd640 1 -- 172.21.15.103:0/3251568855 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f53dc0287e0 con 0x55dd45491760 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53caffd640 1 -- 172.21.15.103:0/3251568855 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53dc04d050 con 0x55dd45491760 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53f443c740 10 client.25031 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53f443c740 1 -- 172.21.15.103:0/3251568855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dd45520200 con 0x55dd45491760 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53caffd640 1 -- 172.21.15.103:0/3251568855 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53dc040460 con 0x55dd45491760 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53caffd640 1 -- 172.21.15.103:0/3251568855 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f53dc01b040 con 0x55dd45491760 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53f443c740 10 client.25031 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53f443c740 10 client.25031 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53f443c740 20 client.25031 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53f443c740 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd455208c0 0x55dd45522ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53f443c740 4 client.25031 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.2, "await_for": 45.0}] 2024-09-05T02:03:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.841+0000 7f53f443c740 1 -- 172.21.15.103:0/3251568855 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.2, "await_for": 45.0}) -- 0x7f53dc03a070 con 0x55dd455208c0 2024-09-05T02:03:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.842+0000 7f53eaffd640 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd455208c0 0x55dd45522ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.842+0000 7f53eaffd640 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd455208c0 0x55dd45522ce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e68670a3ee843a01 server_cookie=70d19a62de990596 in_seq=0 out_seq=0 2024-09-05T02:03:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.842+0000 7f53caffd640 10 client.25031 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.851+0000 7f53caffd640 1 -- 172.21.15.103:0/3251568855 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f53dc03a070 con 0x55dd455208c0 2024-09-05T02:03:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.851+0000 7f53caffd640 10 client.25031 handle_command_reply: tid=0 2024-09-05T02:03:21.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.851+0000 7f53f443c740 1 client.25031 shutdown 2024-09-05T02:03:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.851+0000 7f53f443c740 1 -- 172.21.15.103:0/3251568855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 msgr2=0x55dd454968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.851+0000 7f53f443c740 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd454968e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01e550 tx=0x7f53dc01fcf0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.851+0000 7f53f443c740 1 -- 172.21.15.103:0/3251568855 shutdown_connections 2024-09-05T02:03:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.852+0000 7f53f443c740 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd455208c0 0x55dd45522ce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.852+0000 7f53f443c740 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd4549c940 0x55dd45497360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.852+0000 7f53f443c740 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd45492080 0x55dd45496e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.852+0000 7f53f443c740 1 --2- 172.21.15.103:0/3251568855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd45491760 0x55dd454968e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.852+0000 7f53f443c740 1 -- 172.21.15.103:0/3251568855 >> 172.21.15.103:0/3251568855 conn(0x55dd4548dcf0 msgr2=0x55dd45493200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.852+0000 7f53caffd640 0 client.25031 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.852+0000 7f53f443c740 1 -- 172.21.15.103:0/3251568855 shutdown_connections 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.852+0000 7f53f443c740 1 -- 172.21.15.103:0/3251568855 wait complete. 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.852+0000 7f53f443c740 20 client.25031 trim_cache size 0 max 16384 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 29, 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:03:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 2, 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "9c0651d9": { 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 2, 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 148.2, 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:03:21.855 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:03:21.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:03:21.856 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:03:21.856 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:21.856 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:21.856 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:21.856 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:21.856 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:21.856 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 -- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53d40a7010 msgr2=0x7f53d40a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53d40a7010 0x7f53d40a94d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55dd4548eb50 tx=0x7f53dc00a2d0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 -- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102570 msgr2=0x7f53ec197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102570 0x7f53ec197850 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b520 tx=0x7f53e000b9e0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 -- 172.21.15.103:0/872768931 shutdown_connections 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f53d40a7010 0x7f53d40a94d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53ec102ef0 0x7f53ec19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec102570 0x7f53ec197850 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 --2- 172.21.15.103:0/872768931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec100e20 0x7f53ec197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:21.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.853+0000 7f53f2063640 1 -- 172.21.15.103:0/872768931 >> 172.21.15.103:0/872768931 conn(0x7f53ec0fc7d0 msgr2=0x7f53ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:21.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.854+0000 7f53f2063640 1 -- 172.21.15.103:0/872768931 shutdown_connections 2024-09-05T02:03:21.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:21.854+0000 7f53f2063640 1 -- 172.21.15.103:0/872768931 wait complete. 2024-09-05T02:03:22.572 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:22 smithi103 ceph-mon[30902]: pgmap v378: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-05T02:03:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:22 smithi167 ceph-mon[35122]: pgmap v378: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-05T02:03:22.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:22 smithi139 ceph-mon[34262]: pgmap v378: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-05T02:03:22.759 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9c0651d9', quiesce took 5.0 seconds. Will release after: 23.2 2024-09-05T02:03:24.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:24 smithi167 ceph-mon[35122]: pgmap v379: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:03:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:24 smithi139 ceph-mon[34262]: pgmap v379: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:03:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:24 smithi103 ceph-mon[30902]: pgmap v379: 129 pgs: 129 active+clean; 673 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:03:25.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:25 smithi167 ceph-mon[35122]: pgmap v380: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-05T02:03:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:25 smithi139 ceph-mon[34262]: pgmap v380: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-05T02:03:25.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:25 smithi103 ceph-mon[30902]: pgmap v380: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-05T02:03:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:28 smithi167 ceph-mon[35122]: pgmap v381: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T02:03:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:28 smithi139 ceph-mon[34262]: pgmap v381: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T02:03:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:28 smithi103 ceph-mon[30902]: pgmap v381: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T02:03:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:30 smithi167 ceph-mon[35122]: pgmap v382: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-05T02:03:30.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:30 smithi139 ceph-mon[34262]: pgmap v382: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-05T02:03:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:30 smithi103 ceph-mon[30902]: pgmap v382: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-05T02:03:32.451 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:32 smithi103 ceph-mon[30902]: pgmap v383: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-05T02:03:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:32 smithi167 ceph-mon[35122]: pgmap v383: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-05T02:03:32.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:32 smithi139 ceph-mon[34262]: pgmap v383: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-05T02:03:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.868+0000 7fd3d115f640 1 -- 172.21.15.103:0/20685917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc101d80 msgr2=0x7fd3cc108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.868+0000 7fd3d115f640 1 --2- 172.21.15.103:0/20685917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc101d80 0x7fd3cc108610 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009ca0 tx=0x7fd3c402b690 comp rx=0 tx=0).stop 2024-09-05T02:03:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.868+0000 7fd3d115f640 1 -- 172.21.15.103:0/20685917 shutdown_connections 2024-09-05T02:03:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.868+0000 7fd3d115f640 1 --2- 172.21.15.103:0/20685917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc101d80 0x7fd3cc108610 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.868+0000 7fd3d115f640 1 --2- 172.21.15.103:0/20685917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc1013c0 0x7fd3cc101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.868+0000 7fd3d115f640 1 --2- 172.21.15.103:0/20685917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc1001d0 0x7fd3cc1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.868+0000 7fd3d115f640 1 -- 172.21.15.103:0/20685917 >> 172.21.15.103:0/20685917 conn(0x7fd3cc0fb930 msgr2=0x7fd3cc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.868+0000 7fd3d115f640 1 -- 172.21.15.103:0/20685917 shutdown_connections 2024-09-05T02:03:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.869+0000 7fd3d115f640 1 -- 172.21.15.103:0/20685917 wait complete. 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.870+0000 7fd3d115f640 1 Processor -- start 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.870+0000 7fd3d115f640 1 -- start start 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.870+0000 7fd3d115f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc1001d0 0x7fd3cc195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.870+0000 7fd3d115f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc1013c0 0x7fd3cc195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.870+0000 7fd3d115f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc101d80 0x7fd3cc19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3d115f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc10b4f0 con 0x7fd3cc1001d0 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3cad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc1001d0 0x7fd3cc195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3d115f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc10b370 con 0x7fd3cc1013c0 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3cad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc1001d0 0x7fd3cc195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33092/0 (socket says 172.21.15.103:33092) 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3cad9d640 1 -- 172.21.15.103:0/336758792 learned_addr learned my addr 172.21.15.103:0/336758792 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc101d80 0x7fd3cc19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3ca59c640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc1013c0 0x7fd3cc195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3d115f640 1 -- 172.21.15.103:0/336758792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc10b670 con 0x7fd3cc101d80 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3cb59e640 1 -- 172.21.15.103:0/336758792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc1013c0 msgr2=0x7fd3cc195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.871+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc1013c0 0x7fd3cc195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3cb59e640 1 -- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc1001d0 msgr2=0x7fd3cc195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc1001d0 0x7fd3cc195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3cb59e640 1 -- 172.21.15.103:0/336758792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc19d570 con 0x7fd3cc101d80 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3cad9d640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc1001d0 0x7fd3cc195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc101d80 0x7fd3cc19cf00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3c402bbb0 tx=0x7fd3c4032400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3b3fff640 1 -- 172.21.15.103:0/336758792 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4041070 con 0x7fd3cc101d80 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3b3fff640 1 -- 172.21.15.103:0/336758792 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c40309c0 con 0x7fd3cc101d80 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3d115f640 1 -- 172.21.15.103:0/336758792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3cc19d800 con 0x7fd3cc101d80 2024-09-05T02:03:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3b3fff640 1 -- 172.21.15.103:0/336758792 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4031610 con 0x7fd3cc101d80 2024-09-05T02:03:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.872+0000 7fd3d115f640 1 -- 172.21.15.103:0/336758792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3cc19dcf0 con 0x7fd3cc101d80 2024-09-05T02:03:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.876+0000 7fd3d3fd5740 1 Processor -- start 2024-09-05T02:03:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.876+0000 7fd3b3fff640 1 -- 172.21.15.103:0/336758792 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd3c403a040 con 0x7fd3cc101d80 2024-09-05T02:03:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.876+0000 7fd3b3fff640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3b40a2c30 0x7fd3b40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3d3fd5740 1 -- start start 2024-09-05T02:03:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3b3fff640 1 -- 172.21.15.103:0/336758792 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd3c40f1840 con 0x7fd3cc101d80 2024-09-05T02:03:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387ba1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387ba6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387ba9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565387b052f0 con 0x565387ba1990 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565387aad270 con 0x565387ba22d0 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565387b012b0 con 0x565387ba7360 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3ca59c640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3b40a2c30 0x7fd3b40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3cad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387ba1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3cb59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387ba9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3cb59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387ba9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:38458/0 (socket says 172.21.15.103:38458) 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.877+0000 7fd3cb59e640 1 -- 172.21.15.103:0/682784693 learned_addr learned my addr 172.21.15.103:0/682784693 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.878+0000 7fd3cb59e640 1 -- 172.21.15.103:0/682784693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 msgr2=0x565387ba6d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T02:03:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.878+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/682784693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387ba6d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.878+0000 7fd3cb59e640 1 -- 172.21.15.103:0/682784693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 msgr2=0x565387ba1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.878+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/682784693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387ba1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.878+0000 7fd3ca59c640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3b40a2c30 0x7fd3b40a50f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc196e60 tx=0x7fd3c0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.878+0000 7fd3cb59e640 1 -- 172.21.15.103:0/682784693 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565387aac340 con 0x565387ba7360 2024-09-05T02:03:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.878+0000 7fd3cad9d640 1 --2- 172.21.15.103:0/682784693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387ba1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.878+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/682784693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387ba9820 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40c63d0 tx=0x7fd3c40bda60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=344373c69fbec3c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.878+0000 7fd3b17fa640 1 -- 172.21.15.103:0/682784693 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40db070 con 0x565387ba7360 2024-09-05T02:03:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.879+0000 7fd3b17fa640 1 -- 172.21.15.103:0/682784693 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c40be390 con 0x565387ba7360 2024-09-05T02:03:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3b17fa640 1 -- 172.21.15.103:0/682784693 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4004040 con 0x565387ba7360 2024-09-05T02:03:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/682784693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 msgr2=0x565387ba9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/682784693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387ba9820 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40c63d0 tx=0x7fd3c40bda60 comp rx=0 tx=0).stop 2024-09-05T02:03:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/682784693 shutdown_connections 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/682784693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387ba9820 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/682784693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387ba6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/682784693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387ba1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/682784693 >> 172.21.15.103:0/682784693 conn(0x565387b9d000 msgr2=0x565387b9d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/682784693 shutdown_connections 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/682784693 wait complete. 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.880+0000 7fd3d3fd5740 1 -- start start 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3d3fd5740 1 Processor -- start 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387bbc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387bbcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387bc3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565387bbb100 con 0x565387ba1990 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565387a8e290 con 0x565387ba22d0 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565387bbb280 con 0x565387ba7360 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3d3fd5740 10 client.0 _ll_register_callbacks cb 0x565387abf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3cb59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387bc3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3cad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387bbc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3cb59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387bc3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:38464/0 (socket says 172.21.15.103:38464) 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3cb59e640 1 -- 172.21.15.103:0/2371163056 learned_addr learned my addr 172.21.15.103:0/2371163056 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.881+0000 7fd3ca59c640 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387bbcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3cb59e640 1 -- 172.21.15.103:0/2371163056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 msgr2=0x565387bbcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387bbcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3cb59e640 1 -- 172.21.15.103:0/2371163056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 msgr2=0x565387bbc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387bbc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3cb59e640 1 -- 172.21.15.103:0/2371163056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565387818210 con 0x565387ba7360 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3cad9d640 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387bbc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387bc3f40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40bdc20 tx=0x7fd3c40bdc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1968e4ad0556fcb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3aeff5640 1 -- 172.21.15.103:0/2371163056 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40db070 con 0x565387ba7360 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3aeff5640 1 -- 172.21.15.103:0/2371163056 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd3c4030b70 con 0x565387ba7360 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.882+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2371163056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c40be390 con 0x565387ba7360 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3aeff5640 1 -- 172.21.15.103:0/2371163056 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3c4030d70 con 0x565387ba7360 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3d3fd5740 10 client.25000 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2371163056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653878e4620 con 0x565387ba7360 2024-09-05T02:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3aeff5640 1 -- 172.21.15.103:0/2371163056 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c40be390 con 0x565387ba7360 2024-09-05T02:03:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3aeff5640 1 -- 172.21.15.103:0/2371163056 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd3c40d5030 con 0x565387ba7360 2024-09-05T02:03:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3d3fd5740 10 client.25000 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3d3fd5740 10 client.25000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:03:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3d3fd5740 20 client.25000 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565387bc8240 0x565387bca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3d3fd5740 4 client.25000 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.883+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2371163056 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565387b12a20 con 0x565387bc8240 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.884+0000 7fd3cad9d640 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565387bc8240 0x565387bca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.884+0000 7fd3aeff5640 10 client.25000 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.884+0000 7fd3cad9d640 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565387bc8240 0x565387bca630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab44a3a521977990 server_cookie=f935a9d27f6da01c in_seq=0 out_seq=0 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3aeff5640 1 -- 172.21.15.103:0/2371163056 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565387b12a20 con 0x565387bc8240 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3aeff5640 10 client.25000 handle_command_reply: tid=0 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 client.25000 shutdown 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2371163056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 msgr2=0x565387bc3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387bc3f40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40bdc20 tx=0x7fd3c40bdc90 comp rx=0 tx=0).stop 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2371163056 shutdown_connections 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565387bc8240 0x565387bca630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba7360 0x565387bc3f40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387bbcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2371163056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba1990 0x565387bbc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2371163056 >> 172.21.15.103:0/2371163056 conn(0x565387b9d000 msgr2=0x565387b9e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3aeff5640 0 client.25000 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2371163056 shutdown_connections 2024-09-05T02:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2371163056 wait complete. 2024-09-05T02:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.885+0000 7fd3d3fd5740 20 client.25000 trim_cache size 0 max 16384 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.913+0000 7fd3d3fd5740 1 Processor -- start 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.913+0000 7fd3d3fd5740 1 -- start start 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba1990 0x565387b9ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387bc3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387b9e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc19fc10 con 0x565387ba7360 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1a1750 con 0x565387ba22d0 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3cb59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387b9e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3cb59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387b9e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33130/0 (socket says 172.21.15.103:33130) 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3cb59e640 1 -- 172.21.15.103:0/2296832631 learned_addr learned my addr 172.21.15.103:0/2296832631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3cad9d640 1 --2- 172.21.15.103:0/2296832631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba1990 0x565387b9ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1a02e0 con 0x565387ba1990 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.914+0000 7fd3ca59c640 1 --2- 172.21.15.103:0/2296832631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387bc3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.915+0000 7fd3cb59e640 1 -- 172.21.15.103:0/2296832631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba1990 msgr2=0x565387b9ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.915+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/2296832631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba1990 0x565387b9ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.915+0000 7fd3cb59e640 1 -- 172.21.15.103:0/2296832631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 msgr2=0x565387bc3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.915+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/2296832631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387bc3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.915+0000 7fd3cb59e640 1 -- 172.21.15.103:0/2296832631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565387bbb3d0 con 0x565387ba7360 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.915+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/2296832631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387b9e5c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40ba660 tx=0x7fd3c4032a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf906ee7e8842c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.915+0000 7fd3b0ff9640 1 -- 172.21.15.103:0/2296832631 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40db070 con 0x565387ba7360 2024-09-05T02:03:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.915+0000 7fd3b0ff9640 1 -- 172.21.15.103:0/2296832631 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c40be390 con 0x565387ba7360 2024-09-05T02:03:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.915+0000 7fd3b0ff9640 1 -- 172.21.15.103:0/2296832631 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40c2040 con 0x565387ba7360 2024-09-05T02:03:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.916+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2296832631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 msgr2=0x565387b9e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.916+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2296832631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387b9e5c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40ba660 tx=0x7fd3c4032a00 comp rx=0 tx=0).stop 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.916+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2296832631 shutdown_connections 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.916+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2296832631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387b9e5c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.916+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2296832631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba22d0 0x565387bc3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.916+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/2296832631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba1990 0x565387b9ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.916+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2296832631 >> 172.21.15.103:0/2296832631 conn(0x565387b9d000 msgr2=0x565387b9deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.916+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2296832631 shutdown_connections 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.916+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/2296832631 wait complete. 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3d3fd5740 1 -- start start 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3d3fd5740 1 Processor -- start 2024-09-05T02:03:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba1990 0x565387bc1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba22d0 0x565387bc18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3d3fd5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387bc28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1a1080 con 0x565387ba7360 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1a09b0 con 0x565387ba1990 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3d3fd5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc074150 con 0x565387ba22d0 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3d3fd5740 10 client.0 _ll_register_callbacks cb 0x7fd3c40d9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3cad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba1990 0x565387bc1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3cad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba1990 0x565387bc1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:44522/0 (socket says 172.21.15.103:44522) 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3cad9d640 1 -- 172.21.15.103:0/1949925671 learned_addr learned my addr 172.21.15.103:0/1949925671 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387bc28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.917+0000 7fd3ca59c640 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba22d0 0x565387bc18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3cad9d640 1 -- 172.21.15.103:0/1949925671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba22d0 msgr2=0x565387bc18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3cad9d640 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba22d0 0x565387bc18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3cad9d640 1 -- 172.21.15.103:0/1949925671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 msgr2=0x565387bc28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3cad9d640 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387bc28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3cad9d640 1 -- 172.21.15.103:0/1949925671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565387ba2f60 con 0x565387ba1990 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3cb59e640 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387bc28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3cad9d640 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba1990 0x565387bc1370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x565387b9de80 tx=0x7fd3bc012d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d2206f35567099c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3aeff5640 1 -- 172.21.15.103:0/1949925671 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc01ee00 con 0x565387ba1990 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/1949925671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c40be390 con 0x565387ba1990 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.918+0000 7fd3aeff5640 1 -- 172.21.15.103:0/1949925671 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd3bc025400 con 0x565387ba1990 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3aeff5640 1 -- 172.21.15.103:0/1949925671 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3bc033050 con 0x565387ba1990 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3d3fd5740 10 client.25043 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/1949925671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565387bc4110 con 0x565387ba1990 2024-09-05T02:03:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3aeff5640 1 -- 172.21.15.103:0/1949925671 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c40be390 con 0x565387ba1990 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3aeff5640 1 -- 172.21.15.103:0/1949925671 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd3bc010730 con 0x565387ba1990 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3d3fd5740 10 client.25043 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3d3fd5740 10 client.25043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3d3fd5740 20 client.25043 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565387c378a0 0x565387c39cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3d3fd5740 4 client.25043 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/1949925671 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565387b0cd70 con 0x565387c378a0 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.919+0000 7fd3ca59c640 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565387c378a0 0x565387c39cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.920+0000 7fd3aeff5640 10 client.25043 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.920+0000 7fd3ca59c640 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565387c378a0 0x565387c39cc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a3c6e5922246fb server_cookie=b24ff2b57451cd0f in_seq=0 out_seq=0 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3aeff5640 1 -- 172.21.15.103:0/1949925671 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565387b0cd70 con 0x565387c378a0 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3aeff5640 10 client.25043 handle_command_reply: tid=0 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 client.25043 shutdown 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/1949925671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba1990 msgr2=0x565387bc1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba1990 0x565387bc1370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x565387b9de80 tx=0x7fd3bc012d90 comp rx=0 tx=0).stop 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/1949925671 shutdown_connections 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x565387c378a0 0x565387c39cc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565387ba7360 0x565387bc28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565387ba22d0 0x565387bc18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 --2- 172.21.15.103:0/1949925671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565387ba1990 0x565387bc1370 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/1949925671 >> 172.21.15.103:0/1949925671 conn(0x565387b9d000 msgr2=0x565387b9e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3aeff5640 0 client.25043 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/1949925671 shutdown_connections 2024-09-05T02:03:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 1 -- 172.21.15.103:0/1949925671 wait complete. 2024-09-05T02:03:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.924+0000 7fd3d3fd5740 20 client.25043 trim_cache size 0 max 16384 2024-09-05T02:03:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:03:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:03:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:03:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:03:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 -- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3b40a2c30 msgr2=0x7fd3b40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3b40a2c30 0x7fd3b40a50f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc196e60 tx=0x7fd3c0009110 comp rx=0 tx=0).stop 2024-09-05T02:03:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 -- 172.21.15.103:0/336758792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc101d80 msgr2=0x7fd3cc19cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc101d80 0x7fd3cc19cf00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3c402bbb0 tx=0x7fd3c4032400 comp rx=0 tx=0).stop 2024-09-05T02:03:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 -- 172.21.15.103:0/336758792 shutdown_connections 2024-09-05T02:03:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd3b40a2c30 0x7fd3b40a50f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc101d80 0x7fd3cc19cf00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc1013c0 0x7fd3cc195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 --2- 172.21.15.103:0/336758792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3cc1001d0 0x7fd3cc195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 -- 172.21.15.103:0/336758792 >> 172.21.15.103:0/336758792 conn(0x7fd3cc0fb930 msgr2=0x7fd3cc0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 -- 172.21.15.103:0/336758792 shutdown_connections 2024-09-05T02:03:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:33.925+0000 7fd3d115f640 1 -- 172.21.15.103:0/336758792 wait complete. 2024-09-05T02:03:34.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:34 smithi139 ceph-mon[34262]: pgmap v384: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T02:03:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:34 smithi167 ceph-mon[35122]: pgmap v384: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T02:03:34.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:34 smithi103 ceph-mon[30902]: pgmap v384: 129 pgs: 129 active+clean; 674 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T02:03:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:36 smithi167 ceph-mon[35122]: pgmap v385: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-05T02:03:36.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:36 smithi139 ceph-mon[34262]: pgmap v385: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-05T02:03:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:36 smithi103 ceph-mon[30902]: pgmap v385: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-05T02:03:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:38 smithi167 ceph-mon[35122]: pgmap v386: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-05T02:03:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:38 smithi139 ceph-mon[34262]: pgmap v386: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-05T02:03:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:38 smithi103 ceph-mon[30902]: pgmap v386: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-05T02:03:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:40 smithi167 ceph-mon[35122]: pgmap v387: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-05T02:03:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:40 smithi139 ceph-mon[34262]: pgmap v387: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-05T02:03:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:40 smithi103 ceph-mon[30902]: pgmap v387: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-05T02:03:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:42 smithi167 ceph-mon[35122]: pgmap v388: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-05T02:03:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:42 smithi139 ceph-mon[34262]: pgmap v388: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-05T02:03:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:42 smithi103 ceph-mon[30902]: pgmap v388: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-05T02:03:44.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:44 smithi103 ceph-mon[30902]: pgmap v389: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-05T02:03:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:44 smithi167 ceph-mon[35122]: pgmap v389: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-05T02:03:44.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:44 smithi139 ceph-mon[34262]: pgmap v389: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-05T02:03:45.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:03:46.225 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:03:46.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:46 smithi103 ceph-mon[30902]: pgmap v390: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T02:03:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:46 smithi167 ceph-mon[35122]: pgmap v390: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T02:03:46.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:46 smithi139 ceph-mon[34262]: pgmap v390: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T02:03:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.720+0000 7fd064467640 1 -- 172.21.15.103:0/3317764366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c100a80 msgr2=0x7fd05c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.720+0000 7fd064467640 1 --2- 172.21.15.103:0/3317764366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c100a80 0x7fd05c100ee0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009930 tx=0x7fd04c02a420 comp rx=0 tx=0).stop 2024-09-05T02:03:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.720+0000 7fd064467640 1 -- 172.21.15.103:0/3317764366 shutdown_connections 2024-09-05T02:03:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.720+0000 7fd064467640 1 --2- 172.21.15.103:0/3317764366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd05c101420 0x7fd05c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.720+0000 7fd064467640 1 --2- 172.21.15.103:0/3317764366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c100a80 0x7fd05c100ee0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.720+0000 7fd064467640 1 --2- 172.21.15.103:0/3317764366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c106a80 0x7fd05c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.720+0000 7fd064467640 1 -- 172.21.15.103:0/3317764366 >> 172.21.15.103:0/3317764366 conn(0x7fd05c0fc7c0 msgr2=0x7fd05c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.720+0000 7fd064467640 1 -- 172.21.15.103:0/3317764366 shutdown_connections 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.720+0000 7fd064467640 1 -- 172.21.15.103:0/3317764366 wait complete. 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd064467640 1 Processor -- start 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd064467640 1 -- start start 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd064467640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd05c100a80 0x7fd05c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd064467640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c101420 0x7fd05c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd064467640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c106a80 0x7fd05c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd064467640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c110d40 con 0x7fd05c101420 2024-09-05T02:03:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd064467640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c110bc0 con 0x7fd05c100a80 2024-09-05T02:03:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd064467640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c110ec0 con 0x7fd05c106a80 2024-09-05T02:03:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd061a02640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c101420 0x7fd05c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd061a02640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c101420 0x7fd05c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35474/0 (socket says 172.21.15.103:35474) 2024-09-05T02:03:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd061a02640 1 -- 172.21.15.103:0/3579529984 learned_addr learned my addr 172.21.15.103:0/3579529984 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd062a04640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c106a80 0x7fd05c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd062203640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd05c100a80 0x7fd05c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd061a02640 1 -- 172.21.15.103:0/3579529984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c106a80 msgr2=0x7fd05c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd061a02640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c106a80 0x7fd05c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd061a02640 1 -- 172.21.15.103:0/3579529984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd05c100a80 msgr2=0x7fd05c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd061a02640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd05c100a80 0x7fd05c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd061a02640 1 -- 172.21.15.103:0/3579529984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c19d360 con 0x7fd05c101420 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.721+0000 7fd062203640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd05c100a80 0x7fd05c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.722+0000 7fd061a02640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c101420 0x7fd05c1988f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005ab0 tx=0x7fd04c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.722+0000 7fd0537fe640 1 -- 172.21.15.103:0/3579529984 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c03f070 con 0x7fd05c101420 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.722+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c19d5f0 con 0x7fd05c101420 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.722+0000 7fd0537fe640 1 -- 172.21.15.103:0/3579529984 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd04c031b50 con 0x7fd05c101420 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.722+0000 7fd0537fe640 1 -- 172.21.15.103:0/3579529984 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c035030 con 0x7fd05c101420 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.722+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c1a4f30 con 0x7fd05c101420 2024-09-05T02:03:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.723+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05c1021c0 con 0x7fd05c101420 2024-09-05T02:03:46.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.724+0000 7fd0537fe640 1 -- 172.21.15.103:0/3579529984 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd04c042070 con 0x7fd05c101420 2024-09-05T02:03:46.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.725+0000 7fd0537fe640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0480a2b80 0x7fd0480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:46.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.725+0000 7fd062203640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0480a2b80 0x7fd0480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:46.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.725+0000 7fd0537fe640 1 -- 172.21.15.103:0/3579529984 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd04c0efb00 con 0x7fd05c101420 2024-09-05T02:03:46.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.725+0000 7fd062203640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0480a2b80 0x7fd0480a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd058005e00 tx=0x7fd058005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:46.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.732+0000 7fd0537fe640 1 -- 172.21.15.103:0/3579529984 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd04c0bc6b0 con 0x7fd05c101420 2024-09-05T02:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.936+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd05c10a020 con 0x7fd05c101420 2024-09-05T02:03:46.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.936+0000 7fd0537fe640 1 -- 172.21.15.103:0/3579529984 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7fd04c0bbfb0 con 0x7fd05c101420 2024-09-05T02:03:46.938 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:03:46.939 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":29,"btime":"2024-09-05T02:02:41:223802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:02:41.148036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:03:46.939 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 29 2024-09-05T02:03:46.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0480a2b80 msgr2=0x7fd0480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:46.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0480a2b80 0x7fd0480a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd058005e00 tx=0x7fd058005c70 comp rx=0 tx=0).stop 2024-09-05T02:03:46.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c101420 msgr2=0x7fd05c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:46.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c101420 0x7fd05c1988f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005ab0 tx=0x7fd04c003b90 comp rx=0 tx=0).stop 2024-09-05T02:03:46.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 shutdown_connections 2024-09-05T02:03:46.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd0480a2b80 0x7fd0480a5040 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:46.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c106a80 0x7fd05c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:46.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd05c101420 0x7fd05c1988f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:46.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 --2- 172.21.15.103:0/3579529984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd05c100a80 0x7fd05c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:46.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 >> 172.21.15.103:0/3579529984 conn(0x7fd05c0fc7c0 msgr2=0x7fd05c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:46.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 shutdown_connections 2024-09-05T02:03:46.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:46.939+0000 7fd064467640 1 -- 172.21.15.103:0/3579529984 wait complete. 2024-09-05T02:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.020+0000 7f0d80952640 1 -- 172.21.15.103:0/1444228085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 msgr2=0x7f0d7c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.020+0000 7f0d80952640 1 --2- 172.21.15.103:0/1444228085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 0x7f0d7c1031d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009900 tx=0x7f0d6c02a410 comp rx=0 tx=0).stop 2024-09-05T02:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.020+0000 7f0d80952640 1 -- 172.21.15.103:0/1444228085 shutdown_connections 2024-09-05T02:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.020+0000 7f0d80952640 1 --2- 172.21.15.103:0/1444228085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d7c103710 0x7f0d7c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.020+0000 7f0d80952640 1 --2- 172.21.15.103:0/1444228085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 0x7f0d7c1031d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.020+0000 7f0d80952640 1 --2- 172.21.15.103:0/1444228085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d7c101b60 0x7f0d7c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.020+0000 7f0d80952640 1 -- 172.21.15.103:0/1444228085 >> 172.21.15.103:0/1444228085 conn(0x7f0d7c0fd2e0 msgr2=0x7f0d7c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.020+0000 7f0d80952640 1 -- 172.21.15.103:0/1444228085 shutdown_connections 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.020+0000 7f0d80952640 1 -- 172.21.15.103:0/1444228085 wait complete. 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.021+0000 7f0d80952640 1 Processor -- start 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.021+0000 7f0d80952640 1 -- start start 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.021+0000 7f0d80952640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d7c101b60 0x7f0d7c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.021+0000 7f0d80952640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 0x7f0d7c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.021+0000 7f0d80952640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d7c103710 0x7f0d7c19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.021+0000 7f0d80952640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c10d6e0 con 0x7f0d7c102d50 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.021+0000 7f0d80952640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c10d560 con 0x7f0d7c103710 2024-09-05T02:03:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.021+0000 7f0d80952640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c10d860 con 0x7f0d7c101b60 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 0x7f0d7c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 0x7f0d7c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35480/0 (socket says 172.21.15.103:35480) 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d79d9b640 1 -- 172.21.15.103:0/2577632321 learned_addr learned my addr 172.21.15.103:0/2577632321 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d7c101b60 0x7f0d7c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d7ad9d640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d7c103710 0x7f0d7c19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d79d9b640 1 -- 172.21.15.103:0/2577632321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d7c101b60 msgr2=0x7f0d7c197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d7c101b60 0x7f0d7c197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d79d9b640 1 -- 172.21.15.103:0/2577632321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d7c103710 msgr2=0x7f0d7c19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d7c103710 0x7f0d7c19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d79d9b640 1 -- 172.21.15.103:0/2577632321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c19f930 con 0x7f0d7c102d50 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 0x7f0d7c198260 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0314e0 tx=0x7f0d6c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2577632321 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c03f070 con 0x7f0d7c102d50 2024-09-05T02:03:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.022+0000 7f0d80952640 1 -- 172.21.15.103:0/2577632321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c19fbc0 con 0x7f0d7c102d50 2024-09-05T02:03:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.023+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2577632321 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d6c031c40 con 0x7f0d7c102d50 2024-09-05T02:03:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.023+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2577632321 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c035030 con 0x7f0d7c102d50 2024-09-05T02:03:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.023+0000 7f0d80952640 1 -- 172.21.15.103:0/2577632321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c1a00b0 con 0x7f0d7c102d50 2024-09-05T02:03:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.025+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2577632321 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0d6c042070 con 0x7f0d7c102d50 2024-09-05T02:03:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.025+0000 7f0d837cd740 1 Processor -- start 2024-09-05T02:03:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.025+0000 7f0d837cd740 1 -- start start 2024-09-05T02:03:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e64da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e652e0 0x5618a5e69da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 0x5618a5e6c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d837cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618a5dc82f0 con 0x5618a5e649a0 2024-09-05T02:03:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d837cd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5618a5d70270 con 0x5618a5e6a370 2024-09-05T02:03:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d837cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5618a5dc42b0 con 0x5618a5e652e0 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e64da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e64da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35496/0 (socket says 172.21.15.103:35496) 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7ad9d640 1 -- 172.21.15.103:0/3630134193 learned_addr learned my addr 172.21.15.103:0/3630134193 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 0x5618a5e6c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e652e0 0x5618a5e69da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7ad9d640 1 -- 172.21.15.103:0/3630134193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e652e0 msgr2=0x5618a5e69da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7ad9d640 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e652e0 0x5618a5e69da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7ad9d640 1 -- 172.21.15.103:0/3630134193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 msgr2=0x5618a5e6c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7ad9d640 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 0x5618a5e6c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d6b7fe640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d640a2c30 0x7f0d640a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7ad9d640 1 -- 172.21.15.103:0/3630134193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618a5d6f340 con 0x5618a5e649a0 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 0x5618a5e6c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d640a2c30 0x7f0d640a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2577632321 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0d6c0efd40 con 0x7f0d7c102d50 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.026+0000 7f0d7ad9d640 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e64da0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009f40 tx=0x7f0d74010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d98553341cac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d68ff9640 1 -- 172.21.15.103:0/3630134193 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74003b70 con 0x5618a5e649a0 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d68ff9640 1 -- 172.21.15.103:0/3630134193 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d74003d10 con 0x5618a5e649a0 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d68ff9640 1 -- 172.21.15.103:0/3630134193 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74011a10 con 0x5618a5e649a0 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d837cd740 1 -- 172.21.15.103:0/3630134193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 msgr2=0x5618a5e64da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e64da0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009f40 tx=0x7f0d74010860 comp rx=0 tx=0).stop 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d640a2c30 0x7f0d640a50f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d70005d00 tx=0x7f0d70005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d837cd740 1 -- 172.21.15.103:0/3630134193 shutdown_connections 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 0x5618a5e6c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e652e0 0x5618a5e69da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3630134193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e64da0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d837cd740 1 -- 172.21.15.103:0/3630134193 >> 172.21.15.103:0/3630134193 conn(0x5618a5e60010 msgr2=0x5618a5e60410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d837cd740 1 -- 172.21.15.103:0/3630134193 shutdown_connections 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.027+0000 7f0d837cd740 1 -- 172.21.15.103:0/3630134193 wait complete. 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d837cd740 1 -- start start 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d837cd740 1 Processor -- start 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e6ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e652e0 0x5618a5e6f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 0x5618a5e76150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d837cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618a5e6d340 con 0x5618a5e649a0 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d837cd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5618a5d51290 con 0x5618a5e652e0 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d837cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5618a5e6d4c0 con 0x5618a5e6a370 2024-09-05T02:03:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d837cd740 10 client.0 _ll_register_callbacks cb 0x5618a5d82140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e6ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e6ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35510/0 (socket says 172.21.15.103:35510) 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d7a59c640 1 -- 172.21.15.103:0/3813145978 learned_addr learned my addr 172.21.15.103:0/3813145978 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e652e0 0x5618a5e6f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.028+0000 7f0d7ad9d640 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 0x5618a5e76150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d7a59c640 1 -- 172.21.15.103:0/3813145978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 msgr2=0x5618a5e76150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 0x5618a5e76150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d7a59c640 1 -- 172.21.15.103:0/3813145978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e652e0 msgr2=0x5618a5e6f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e652e0 0x5618a5e6f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d7a59c640 1 -- 172.21.15.103:0/3813145978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618a5adb210 con 0x5618a5e649a0 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e652e0 0x5618a5e6f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d7ad9d640 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 0x5618a5e76150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e6ebd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c102bb0 tx=0x7f0d70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f87363e8dddd0b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3813145978 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70023070 con 0x5618a5e649a0 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3813145978 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0d7001f860 con 0x5618a5e649a0 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d837cd740 1 -- 172.21.15.103:0/3813145978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d74003d10 con 0x5618a5e649a0 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3813145978 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d7002f050 con 0x5618a5e649a0 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d837cd740 10 client.15573 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d837cd740 1 -- 172.21.15.103:0/3813145978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618a5ba7620 con 0x5618a5e649a0 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.029+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3813145978 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d74003d10 con 0x5618a5e649a0 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3813145978 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0d7000f040 con 0x5618a5e649a0 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d837cd740 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d837cd740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d837cd740 20 client.15573 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5618a5e7a840 0x5618a5e7cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d837cd740 4 client.15573 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:03:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d837cd740 1 -- 172.21.15.103:0/3813145978 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618a5dd5a20 con 0x5618a5e7a840 2024-09-05T02:03:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5618a5e7a840 0x5618a5e7cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5618a5e7a840 0x5618a5e7cce0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26448b06a2f52dee server_cookie=762fb65d141e0f5b in_seq=0 out_seq=0 2024-09-05T02:03:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.030+0000 7f0d61ffb640 10 client.15573 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3813145978 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618a5dd5a20 con 0x5618a5e7a840 2024-09-05T02:03:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d61ffb640 10 client.15573 handle_command_reply: tid=0 2024-09-05T02:03:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 client.15573 shutdown 2024-09-05T02:03:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 -- 172.21.15.103:0/3813145978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 msgr2=0x5618a5e6ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e6ebd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c102bb0 tx=0x7f0d70004730 comp rx=0 tx=0).stop 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 -- 172.21.15.103:0/3813145978 shutdown_connections 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5618a5e7a840 0x5618a5e7cce0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 0x5618a5e76150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d61ffb640 0 client.15573 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e652e0 0x5618a5e6f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3813145978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e649a0 0x5618a5e6ebd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 -- 172.21.15.103:0/3813145978 >> 172.21.15.103:0/3813145978 conn(0x5618a5e60010 msgr2=0x5618a5e61820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 -- 172.21.15.103:0/3813145978 shutdown_connections 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 1 -- 172.21.15.103:0/3813145978 wait complete. 2024-09-05T02:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.031+0000 7f0d837cd740 20 client.15573 trim_cache size 0 max 16384 2024-09-05T02:03:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d837cd740 1 Processor -- start 2024-09-05T02:03:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d837cd740 1 -- start start 2024-09-05T02:03:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e649a0 0x5618a5e5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e76070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 0x5618a5e61400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d837cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1aa690 con 0x5618a5e652e0 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d837cd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1ac1d0 con 0x5618a5e649a0 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d837cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1aad60 con 0x5618a5e6a370 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e76070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e76070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35514/0 (socket says 172.21.15.103:35514) 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d79d9b640 1 -- 172.21.15.103:0/3661267586 learned_addr learned my addr 172.21.15.103:0/3661267586 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d7ad9d640 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 0x5618a5e61400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e649a0 0x5618a5e5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d79d9b640 1 -- 172.21.15.103:0/3661267586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 msgr2=0x5618a5e61400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 0x5618a5e61400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d79d9b640 1 -- 172.21.15.103:0/3661267586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e649a0 msgr2=0x5618a5e5f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e649a0 0x5618a5e5f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d79d9b640 1 -- 172.21.15.103:0/3661267586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618a5e6e710 con 0x5618a5e652e0 2024-09-05T02:03:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.046+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e649a0 0x5618a5e5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:03:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.047+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e76070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005ec0 tx=0x7f0d6c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07f0bdfe7cf7eec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.047+0000 7f0d63fff640 1 -- 172.21.15.103:0/3661267586 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c049040 con 0x5618a5e652e0 2024-09-05T02:03:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.047+0000 7f0d63fff640 1 -- 172.21.15.103:0/3661267586 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d6c0bc1f0 con 0x5618a5e652e0 2024-09-05T02:03:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.047+0000 7f0d63fff640 1 -- 172.21.15.103:0/3661267586 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0c0170 con 0x5618a5e652e0 2024-09-05T02:03:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.047+0000 7f0d837cd740 1 -- 172.21.15.103:0/3661267586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 msgr2=0x5618a5e76070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.047+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e76070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005ec0 tx=0x7f0d6c0dc040 comp rx=0 tx=0).stop 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 -- 172.21.15.103:0/3661267586 shutdown_connections 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e6a370 0x5618a5e61400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e76070 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3661267586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e649a0 0x5618a5e5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 -- 172.21.15.103:0/3661267586 >> 172.21.15.103:0/3661267586 conn(0x5618a5e60010 msgr2=0x5618a5e5e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 -- 172.21.15.103:0/3661267586 shutdown_connections 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 -- 172.21.15.103:0/3661267586 wait complete. 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 -- start start 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 Processor -- start 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e649a0 0x5618a5e736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e73bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 0x5618a5e74c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1abb00 con 0x5618a5e652e0 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1ab430 con 0x5618a5e6a370 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1a8670 con 0x5618a5e649a0 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d837cd740 10 client.0 _ll_register_callbacks cb 0x7f0d70016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e73bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e73bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35530/0 (socket says 172.21.15.103:35530) 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d79d9b640 1 -- 172.21.15.103:0/3495419823 learned_addr learned my addr 172.21.15.103:0/3495419823 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.048+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e649a0 0x5618a5e736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d7ad9d640 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 0x5618a5e74c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d79d9b640 1 -- 172.21.15.103:0/3495419823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e649a0 msgr2=0x5618a5e736b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e649a0 0x5618a5e736b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d79d9b640 1 -- 172.21.15.103:0/3495419823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 msgr2=0x5618a5e74c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 0x5618a5e74c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d79d9b640 1 -- 172.21.15.103:0/3495419823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618a5e75210 con 0x5618a5e652e0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d79d9b640 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e73bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0ed4e0 tx=0x7f0d6c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fda6798f827acbd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3495419823 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c049070 con 0x5618a5e652e0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3495419823 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0d6c0dc0b0 con 0x5618a5e652e0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d837cd740 1 -- 172.21.15.103:0/3495419823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6c0bc1f0 con 0x5618a5e652e0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3495419823 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d6c0c0370 con 0x5618a5e652e0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d837cd740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d837cd740 1 -- 172.21.15.103:0/3495419823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618a5ee5580 con 0x5618a5e652e0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.049+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3495419823 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d6c0bc1f0 con 0x5618a5e652e0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3495419823 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0d6c0bc450 con 0x5618a5e652e0 2024-09-05T02:03:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d837cd740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d837cd740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:03:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d837cd740 20 client.15579 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5618a5ee5cd0 0x5618a5ee80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d837cd740 4 client.15579 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:03:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d837cd740 1 -- 172.21.15.103:0/3495419823 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618a5dcfd70 con 0x5618a5ee5cd0 2024-09-05T02:03:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5618a5ee5cd0 0x5618a5ee80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d61ffb640 10 client.15579 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.050+0000 7f0d7a59c640 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5618a5ee5cd0 0x5618a5ee80f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570fd4e5c4869c63 server_cookie=448ba188c406d7dd in_seq=0 out_seq=0 2024-09-05T02:03:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d61ffb640 1 -- 172.21.15.103:0/3495419823 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618a5dcfd70 con 0x5618a5ee5cd0 2024-09-05T02:03:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d61ffb640 10 client.15579 handle_command_reply: tid=0 2024-09-05T02:03:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 client.15579 shutdown 2024-09-05T02:03:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 -- 172.21.15.103:0/3495419823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 msgr2=0x5618a5e73bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e73bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0ed4e0 tx=0x7f0d6c02a410 comp rx=0 tx=0).stop 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 -- 172.21.15.103:0/3495419823 shutdown_connections 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5618a5ee5cd0 0x5618a5ee80f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618a5e6a370 0x5618a5e74c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d61ffb640 0 client.15579 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618a5e652e0 0x5618a5e73bf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 --2- 172.21.15.103:0/3495419823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618a5e649a0 0x5618a5e736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 -- 172.21.15.103:0/3495419823 >> 172.21.15.103:0/3495419823 conn(0x5618a5e60010 msgr2=0x5618a5e5f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 -- 172.21.15.103:0/3495419823 shutdown_connections 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 1 -- 172.21.15.103:0/3495419823 wait complete. 2024-09-05T02:03:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.054+0000 7f0d837cd740 20 client.15579 trim_cache size 0 max 16384 2024-09-05T02:03:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:03:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:03:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:03:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:03:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 -- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d640a2c30 msgr2=0x7f0d640a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d640a2c30 0x7f0d640a50f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d70005d00 tx=0x7f0d70005c70 comp rx=0 tx=0).stop 2024-09-05T02:03:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 -- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 msgr2=0x7f0d7c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 0x7f0d7c198260 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0314e0 tx=0x7f0d6c003c10 comp rx=0 tx=0).stop 2024-09-05T02:03:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 -- 172.21.15.103:0/2577632321 shutdown_connections 2024-09-05T02:03:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d7c103710 0x7f0d7c19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d7c102d50 0x7f0d7c198260 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d640a2c30 0x7f0d640a50f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 --2- 172.21.15.103:0/2577632321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d7c101b60 0x7f0d7c197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 -- 172.21.15.103:0/2577632321 >> 172.21.15.103:0/2577632321 conn(0x7f0d7c0fd2e0 msgr2=0x7f0d7c1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 -- 172.21.15.103:0/2577632321 shutdown_connections 2024-09-05T02:03:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:03:47.055+0000 7f0d80952640 1 -- 172.21.15.103:0/2577632321 wait complete. 2024-09-05T02:03:47.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:47 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3579529984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:03:47.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:47 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3579529984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:03:47.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:47 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3579529984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:03:48.603 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:48 smithi103 ceph-mon[30902]: pgmap v391: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:03:48.632 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 9c0651d9 --release --await' 2024-09-05T02:03:48.632 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 9c0651d9 --release --await 2024-09-05T02:03:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:48 smithi167 ceph-mon[35122]: pgmap v391: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:03:48.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:48 smithi139 ceph-mon[34262]: pgmap v391: 129 pgs: 129 active+clean; 675 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T02:03:48.942 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:03:49.458 INFO:tasks.workunit.client.0.smithi103.stderr:Updating files: 93% (12051/12892) Updating files: 94% (12119/12892) Updating files: 95% (12248/12892) Updating files: 96% (12377/12892) Updating files: 97% (12506/12892) Updating files: 98% (12635/12892) Updating files: 99% (12764/12892) Updating files: 100% (12892/12892) Updating files: 100% (12892/12892), done. 2024-09-05T02:03:49.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.553+0000 7f9e184fb640 1 -- 172.21.15.103:0/2600575308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10102570 msgr2=0x7f9e101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.553+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2600575308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10102570 0x7f9e101029b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009900 tx=0x7f9e0002a480 comp rx=0 tx=0).stop 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.553+0000 7f9e184fb640 1 -- 172.21.15.103:0/2600575308 shutdown_connections 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.553+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2600575308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e10102ef0 0x7f9e1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.553+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2600575308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10102570 0x7f9e101029b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.553+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2600575308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e10100e20 0x7f9e10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.553+0000 7f9e184fb640 1 -- 172.21.15.103:0/2600575308 >> 172.21.15.103:0/2600575308 conn(0x7f9e100fc7d0 msgr2=0x7f9e100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.553+0000 7f9e184fb640 1 -- 172.21.15.103:0/2600575308 shutdown_connections 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.553+0000 7f9e184fb640 1 -- 172.21.15.103:0/2600575308 wait complete. 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e184fb640 1 Processor -- start 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e184fb640 1 -- start start 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e184fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10100e20 0x7f9e100fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e184fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e10102570 0x7f9e100fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e184fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e10102ef0 0x7f9e100fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e184fb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10110ed0 con 0x7f9e10100e20 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e184fb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10110d50 con 0x7f9e10102570 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e184fb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10111050 con 0x7f9e10102ef0 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e16297640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10100e20 0x7f9e100fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e16297640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10100e20 0x7f9e100fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40020/0 (socket says 172.21.15.103:40020) 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e16297640 1 -- 172.21.15.103:0/2290763265 learned_addr learned my addr 172.21.15.103:0/2290763265 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.554+0000 7f9e16a98640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e10102ef0 0x7f9e100fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e15a96640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e10102570 0x7f9e100fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e16297640 1 -- 172.21.15.103:0/2290763265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e10102ef0 msgr2=0x7f9e100fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e16297640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e10102ef0 0x7f9e100fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e16297640 1 -- 172.21.15.103:0/2290763265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e10102570 msgr2=0x7f9e100fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e16297640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e10102570 0x7f9e100fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e16297640 1 -- 172.21.15.103:0/2290763265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e101a69c0 con 0x7f9e10100e20 2024-09-05T02:03:49.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e15a96640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e10102570 0x7f9e100fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e16297640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10100e20 0x7f9e100fcc80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00b4a0 tx=0x7f9e0c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e077fe640 1 -- 172.21.15.103:0/2290763265 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c014070 con 0x7f9e10100e20 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e077fe640 1 -- 172.21.15.103:0/2290763265 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e0c010ac0 con 0x7f9e10100e20 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e184fb640 1 -- 172.21.15.103:0/2290763265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e101a6cb0 con 0x7f9e10100e20 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e077fe640 1 -- 172.21.15.103:0/2290763265 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c017c30 con 0x7f9e10100e20 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.555+0000 7f9e184fb640 1 -- 172.21.15.103:0/2290763265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e101a71e0 con 0x7f9e10100e20 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.557+0000 7f9e077fe640 1 -- 172.21.15.103:0/2290763265 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9e0c017070 con 0x7f9e10100e20 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.558+0000 7f9e077fe640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9dfc0a2c00 0x7f9dfc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.558+0000 7f9e15a96640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9dfc0a2c00 0x7f9dfc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.558+0000 7f9e077fe640 1 -- 172.21.15.103:0/2290763265 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9e0c0cea60 con 0x7f9e10100e20 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.558+0000 7f9e1a8d4740 1 Processor -- start 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e1a8d4740 1 -- start start 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e15a96640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9dfc0a2c00 0x7f9dfc0a50c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9e100fe710 tx=0x7f9e00005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 0x5603286de440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de980 0x5603286e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560328643620 con 0x5603286e9240 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5603285eae50 con 0x5603286de980 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56032863f5e0 con 0x5603286de060 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e16a98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e16a98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286eb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40026/0 (socket says 172.21.15.103:40026) 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e16a98640 1 -- 172.21.15.103:0/4082504800 learned_addr learned my addr 172.21.15.103:0/4082504800 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e16297640 1 --2- 172.21.15.103:0/4082504800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 0x5603286de440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e15a96640 1 --2- 172.21.15.103:0/4082504800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de980 0x5603286e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e16a98640 1 -- 172.21.15.103:0/4082504800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 msgr2=0x5603286de440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e16a98640 1 --2- 172.21.15.103:0/4082504800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 0x5603286de440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e16a98640 1 -- 172.21.15.103:0/4082504800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de980 msgr2=0x5603286e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e16a98640 1 --2- 172.21.15.103:0/4082504800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de980 0x5603286e8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.559+0000 7f9e16a98640 1 -- 172.21.15.103:0/4082504800 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603285dbd40 con 0x5603286e9240 2024-09-05T02:03:49.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.560+0000 7f9e16a98640 1 --2- 172.21.15.103:0/4082504800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286eb630 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800a430 tx=0x7f9e08009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95e1f521f3ebe9f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:49.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.560+0000 7f9e04ff9640 1 -- 172.21.15.103:0/4082504800 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08013790 con 0x5603286e9240 2024-09-05T02:03:49.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.560+0000 7f9e04ff9640 1 -- 172.21.15.103:0/4082504800 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e08019070 con 0x5603286e9240 2024-09-05T02:03:49.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.560+0000 7f9e04ff9640 1 -- 172.21.15.103:0/4082504800 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08014d00 con 0x5603286e9240 2024-09-05T02:03:49.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4082504800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 msgr2=0x5603286eb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:49.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/4082504800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286eb630 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800a430 tx=0x7f9e08009650 comp rx=0 tx=0).stop 2024-09-05T02:03:49.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4082504800 shutdown_connections 2024-09-05T02:03:49.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/4082504800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286eb630 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/4082504800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de980 0x5603286e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/4082504800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 0x5603286de440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4082504800 >> 172.21.15.103:0/4082504800 conn(0x5603286da5f0 msgr2=0x5603286da9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4082504800 shutdown_connections 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4082504800 wait complete. 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 -- start start 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.561+0000 7f9e1a8d4740 1 Processor -- start 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 0x5603286e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286de980 0x5603286e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286e9240 0x5603286e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5603286ebe10 con 0x5603286de980 2024-09-05T02:03:49.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5603285cbf10 con 0x5603286e9240 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5603286ebf90 con 0x5603286de060 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e1a8d4740 10 client.0 _ll_register_callbacks cb 0x5603285efdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e15a96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286de980 0x5603286e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e15a96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286de980 0x5603286e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40038/0 (socket says 172.21.15.103:40038) 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e15a96640 1 -- 172.21.15.103:0/3076678042 learned_addr learned my addr 172.21.15.103:0/3076678042 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e16297640 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 0x5603286e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e16a98640 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286e9240 0x5603286e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e15a96640 1 -- 172.21.15.103:0/3076678042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 msgr2=0x5603286e6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e15a96640 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 0x5603286e6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e15a96640 1 -- 172.21.15.103:0/3076678042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286e9240 msgr2=0x5603286e6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e15a96640 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286e9240 0x5603286e6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e15a96640 1 -- 172.21.15.103:0/3076678042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560328352580 con 0x5603286de980 2024-09-05T02:03:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e16a98640 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286e9240 0x5603286e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9e15a96640 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286de980 0x5603286e6590 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009900 tx=0x7f9e00009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea98368433e910ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.562+0000 7f9df9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9df9ffb640 1 -- 172.21.15.103:0/3076678042 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0004f070 con 0x5603286de980 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9df9ffb640 1 -- 172.21.15.103:0/3076678042 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9e00003800 con 0x5603286de980 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/3076678042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e08019070 con 0x5603286de980 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9df9ffb640 1 -- 172.21.15.103:0/3076678042 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e0005b050 con 0x5603286de980 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9e1a8d4740 10 client.15591 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/3076678042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56032841c960 con 0x5603286de980 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9df9ffb640 1 -- 172.21.15.103:0/3076678042 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e08019070 con 0x5603286de980 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9df9ffb640 1 -- 172.21.15.103:0/3076678042 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9e00049a00 con 0x5603286de980 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9e1a8d4740 10 client.15591 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9e1a8d4740 10 client.15591 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9e1a8d4740 20 client.15591 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:49.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5603286f8d40 0x5603286fb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9e1a8d4740 4 client.15591 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.563+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/3076678042 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560328650b90 con 0x5603286f8d40 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.564+0000 7f9e16297640 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5603286f8d40 0x5603286fb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.564+0000 7f9df9ffb640 10 client.15591 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.564+0000 7f9e16297640 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5603286f8d40 0x5603286fb1a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23fd7bb91dd08e9 server_cookie=88b191bae0e928d0 in_seq=0 out_seq=0 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9df9ffb640 1 -- 172.21.15.103:0/3076678042 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560328650b90 con 0x5603286f8d40 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9df9ffb640 10 client.15591 handle_command_reply: tid=0 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 client.15591 shutdown 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/3076678042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286de980 msgr2=0x5603286e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286de980 0x5603286e6590 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009900 tx=0x7f9e00009260 comp rx=0 tx=0).stop 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/3076678042 shutdown_connections 2024-09-05T02:03:49.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5603286f8d40 0x5603286fb1a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286e9240 0x5603286e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286de980 0x5603286e6590 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/3076678042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de060 0x5603286e6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/3076678042 >> 172.21.15.103:0/3076678042 conn(0x5603286da5f0 msgr2=0x5603286e02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:49.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9df9ffb640 0 client.15591 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:49.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/3076678042 shutdown_connections 2024-09-05T02:03:49.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/3076678042 wait complete. 2024-09-05T02:03:49.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.565+0000 7f9e1a8d4740 20 client.15591 trim_cache size 0 max 16384 2024-09-05T02:03:49.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e1a8d4740 1 Processor -- start 2024-09-05T02:03:49.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e1a8d4740 1 -- start start 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 0x5603286e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 0x560328700a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x560328700fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101b0fe0 con 0x5603286e9240 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101b2b20 con 0x5603286de060 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101b16b0 con 0x5603286de980 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e16a98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x560328700fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e16a98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x560328700fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40044/0 (socket says 172.21.15.103:40044) 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e16a98640 1 -- 172.21.15.103:0/4089099700 learned_addr learned my addr 172.21.15.103:0/4089099700 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e16297640 1 --2- 172.21.15.103:0/4089099700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 0x5603286e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.597+0000 7f9e15a96640 1 --2- 172.21.15.103:0/4089099700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 0x560328700a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.598+0000 7f9e16a98640 1 -- 172.21.15.103:0/4089099700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 msgr2=0x560328700a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.598+0000 7f9e16a98640 1 --2- 172.21.15.103:0/4089099700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 0x560328700a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.598+0000 7f9e16a98640 1 -- 172.21.15.103:0/4089099700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 msgr2=0x5603286e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.598+0000 7f9e16a98640 1 --2- 172.21.15.103:0/4089099700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 0x5603286e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.598+0000 7f9e16a98640 1 -- 172.21.15.103:0/4089099700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603286da420 con 0x5603286e9240 2024-09-05T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.598+0000 7f9e16a98640 1 --2- 172.21.15.103:0/4089099700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x560328700fb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009f30 tx=0x7f9e08009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396569fdfc8e5b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.598+0000 7f9dfbfff640 1 -- 172.21.15.103:0/4089099700 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08013890 con 0x5603286e9240 2024-09-05T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.598+0000 7f9dfbfff640 1 -- 172.21.15.103:0/4089099700 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e08019040 con 0x5603286e9240 2024-09-05T02:03:49.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.598+0000 7f9dfbfff640 1 -- 172.21.15.103:0/4089099700 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08014e00 con 0x5603286e9240 2024-09-05T02:03:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4089099700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 msgr2=0x560328700fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/4089099700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x560328700fb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009f30 tx=0x7f9e08009f60 comp rx=0 tx=0).stop 2024-09-05T02:03:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4089099700 shutdown_connections 2024-09-05T02:03:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/4089099700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x560328700fb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/4089099700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 0x560328700a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/4089099700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 0x5603286e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4089099700 >> 172.21.15.103:0/4089099700 conn(0x5603286da5f0 msgr2=0x5603286e47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4089099700 shutdown_connections 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/4089099700 wait complete. 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 -- start start 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.599+0000 7f9e1a8d4740 1 Processor -- start 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 0x5603286e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 0x5603286e3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e1a8d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101b2450 con 0x5603286e9240 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101b1d80 con 0x5603286de060 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e1a8d4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101aefc0 con 0x5603286de980 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e1a8d4740 10 client.0 _ll_register_callbacks cb 0x7f9e000314e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:03:49.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16a98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16a98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286e3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40056/0 (socket says 172.21.15.103:40056) 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16a98640 1 -- 172.21.15.103:0/2708559147 learned_addr learned my addr 172.21.15.103:0/2708559147 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e15a96640 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 0x5603286e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16297640 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 0x5603286e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16a98640 1 -- 172.21.15.103:0/2708559147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 msgr2=0x5603286e3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16a98640 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 0x5603286e3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16a98640 1 -- 172.21.15.103:0/2708559147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 msgr2=0x5603286e3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16a98640 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 0x5603286e3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16a98640 1 -- 172.21.15.103:0/2708559147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603286ef9e0 con 0x5603286e9240 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16297640 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 0x5603286e3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9e16a98640 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286e3bd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9e08004b90 tx=0x7f9e08014db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2009374279f1633c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:03:49.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9df9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.600+0000 7f9df9ffb640 1 -- 172.21.15.103:0/2708559147 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08014620 con 0x5603286e9240 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9df9ffb640 1 -- 172.21.15.103:0/2708559147 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9e08013690 con 0x5603286e9240 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/2708559147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e08019040 con 0x5603286e9240 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9df9ffb640 1 -- 172.21.15.103:0/2708559147 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e08013890 con 0x5603286e9240 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e1a8d4740 10 client.15597 fetch_fsmap learned FSMap version 29 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/2708559147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560328763ee0 con 0x5603286e9240 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9df9ffb640 1 -- 172.21.15.103:0/2708559147 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e08019040 con 0x5603286e9240 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9df9ffb640 1 -- 172.21.15.103:0/2708559147 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9e08033980 con 0x5603286e9240 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e1a8d4740 10 client.15597 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e1a8d4740 10 client.15597 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:03:49.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e1a8d4740 20 client.15597 populate_metadata read hostname 'smithi103' 2024-09-05T02:03:49.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5603287643c0 0x5603287667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:03:49.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e1a8d4740 4 client.15597 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9c0651d9", "release": true, "await": true}] 2024-09-05T02:03:49.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/2708559147 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9c0651d9", "release": true, "await": true}) -- 0x5603286d5950 con 0x5603287643c0 2024-09-05T02:03:49.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.601+0000 7f9e16297640 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5603287643c0 0x5603287667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:03:49.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.602+0000 7f9df9ffb640 10 client.15597 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:49.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.602+0000 7f9e16297640 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5603287643c0 0x5603287667b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692a975331c0eb1c server_cookie=aad1660707bfcf6e in_seq=0 out_seq=0 2024-09-05T02:03:49.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9df9ffb640 1 -- 172.21.15.103:0/2708559147 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5603286d5950 con 0x5603287643c0 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9df9ffb640 10 client.15597 handle_command_reply: tid=0 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 client.15597 shutdown 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/2708559147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 msgr2=0x5603286e3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286e3bd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9e08004b90 tx=0x7f9e08014db0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/2708559147 shutdown_connections 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5603287643c0 0x5603287667b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9df9ffb640 0 client.15597 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603286e9240 0x5603286e3bd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603286de980 0x5603286e3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 --2- 172.21.15.103:0/2708559147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603286de060 0x5603286e3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/2708559147 >> 172.21.15.103:0/2708559147 conn(0x5603286da5f0 msgr2=0x5603286e6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:49.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/2708559147 shutdown_connections 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 1 -- 172.21.15.103:0/2708559147 wait complete. 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.604+0000 7f9e1a8d4740 20 client.15597 trim_cache size 0 max 16384 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 29, 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 4, 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stdout: "9c0651d9": { 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 4, 2024-09-05T02:03:49.611 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:03:49.612 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:03:49.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:03:49.612 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:03:49.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:03:49.612 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:03:49.612 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 148.2, 2024-09-05T02:03:49.612 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:03:49.612 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:03:49.612 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:03:49.613 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:03:49.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:03:49.613 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:03:49.613 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:49.613 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:49.613 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:49.613 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:49.613 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:03:49.613 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 -- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9dfc0a2c00 msgr2=0x7f9dfc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9dfc0a2c00 0x7f9dfc0a50c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9e100fe710 tx=0x7f9e00005a20 comp rx=0 tx=0).stop 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 -- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10100e20 msgr2=0x7f9e100fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10100e20 0x7f9e100fcc80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00b4a0 tx=0x7f9e0c00b960 comp rx=0 tx=0).stop 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 -- 172.21.15.103:0/2290763265 shutdown_connections 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9dfc0a2c00 0x7f9dfc0a50c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e10102ef0 0x7f9e100fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e10102570 0x7f9e100fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 --2- 172.21.15.103:0/2290763265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10100e20 0x7f9e100fcc80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:03:49.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 -- 172.21.15.103:0/2290763265 >> 172.21.15.103:0/2290763265 conn(0x7f9e100fc7d0 msgr2=0x7f9e1006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:03:49.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 -- 172.21.15.103:0/2290763265 shutdown_connections 2024-09-05T02:03:49.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:03:49.605+0000 7f9e184fb640 1 -- 172.21.15.103:0/2290763265 wait complete. 2024-09-05T02:03:50.199 INFO:tasks.workunit.client.0.smithi103.stderr:Note: switching to '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4'. 2024-09-05T02:03:50.199 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:50.199 INFO:tasks.workunit.client.0.smithi103.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-05T02:03:50.199 INFO:tasks.workunit.client.0.smithi103.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-05T02:03:50.199 INFO:tasks.workunit.client.0.smithi103.stderr:state without impacting any branches by switching back to a branch. 2024-09-05T02:03:50.199 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:50.199 INFO:tasks.workunit.client.0.smithi103.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-05T02:03:50.199 INFO:tasks.workunit.client.0.smithi103.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-05T02:03:50.199 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:50.200 INFO:tasks.workunit.client.0.smithi103.stderr: git switch -c 2024-09-05T02:03:50.200 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:50.200 INFO:tasks.workunit.client.0.smithi103.stderr:Or undo this operation with: 2024-09-05T02:03:50.200 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:50.200 INFO:tasks.workunit.client.0.smithi103.stderr: git switch - 2024-09-05T02:03:50.200 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:50.200 INFO:tasks.workunit.client.0.smithi103.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-05T02:03:50.200 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:50.200 INFO:tasks.workunit.client.0.smithi103.stderr:HEAD is now at 91c8a7e56f7 Merge pull request #59582 from zdover23/wip-doc-2024-09-04-backport-59546-to-squid 2024-09-05T02:03:50.204 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-05T02:03:50.246 INFO:tasks.workunit.client.0.smithi103.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-05T02:03:50.248 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-05T02:03:50.248 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-05T02:03:50.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:50 smithi103 ceph-mon[30902]: pgmap v392: 129 pgs: 129 active+clean; 676 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-05T02:03:50.526 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9c0651d9', total seconds elapsed: 32.8 2024-09-05T02:03:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:50 smithi167 ceph-mon[35122]: pgmap v392: 129 pgs: 129 active+clean; 676 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-05T02:03:50.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:50 smithi139 ceph-mon[34262]: pgmap v392: 129 pgs: 129 active+clean; 676 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-05T02:03:51.102 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-05T02:03:51.175 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-05T02:03:51.215 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-05T02:03:51.216 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-05T02:03:51.217 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-05T02:03:51.256 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-05T02:03:51.259 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T02:03:51.259 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-05T02:03:51.274 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-05T02:03:51.276 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-05T02:03:51.276 DEBUG:teuthology.orchestra.run.smithi103:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-05T02:03:51.345 INFO:tasks.workunit.client.0.smithi103.stdout:getting blogbench 2024-09-05T02:03:51.345 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo 'getting blogbench' 2024-09-05T02:03:51.345 INFO:tasks.workunit.client.0.smithi103.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-05T02:03:51.367 INFO:tasks.workunit.client.0.smithi103.stderr:--2024-09-05 02:03:51-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-05T02:03:51.377 INFO:tasks.workunit.client.0.smithi103.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-05T02:03:51.403 INFO:tasks.workunit.client.0.smithi103.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-05T02:03:51.428 INFO:tasks.workunit.client.0.smithi103.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-05T02:03:51.428 INFO:tasks.workunit.client.0.smithi103.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-05T02:03:51.430 INFO:tasks.workunit.client.0.smithi103.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-05T02:03:51.433 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:51.479 INFO:tasks.workunit.client.0.smithi103.stderr: 0K .......... .......... .......... .......... .......... 49% 1.00M 0s 2024-09-05T02:03:51.530 INFO:tasks.workunit.client.0.smithi103.stderr: 50K .......... .......... .......... .......... .......... 99% 1.87M 0s 2024-09-05T02:03:51.530 INFO:tasks.workunit.client.0.smithi103.stderr: 100K 100% 38.0K=0.1s 2024-09-05T02:03:51.530 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:51.530 INFO:tasks.workunit.client.0.smithi103.stderr:2024-09-05 02:03:51 (1015 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-05T02:03:51.531 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-05T02:03:51.531 INFO:tasks.workunit.client.0.smithi103.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-05T02:03:51.564 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/ 2024-09-05T02:03:51.566 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/m4/ 2024-09-05T02:03:51.568 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-05T02:03:51.569 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/man/ 2024-09-05T02:03:51.571 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/man/blogbench.8 2024-09-05T02:03:51.573 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/man/Makefile.am 2024-09-05T02:03:51.576 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/man/Makefile.in 2024-09-05T02:03:51.578 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/ 2024-09-05T02:03:51.580 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-05T02:03:51.582 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-05T02:03:51.585 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/blogbench.c 2024-09-05T02:03:51.587 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/blogbench.h 2024-09-05T02:03:51.589 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-05T02:03:51.592 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-05T02:03:51.593 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-05T02:03:51.596 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-05T02:03:51.598 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-05T02:03:51.601 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/Makefile.am 2024-09-05T02:03:51.603 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/Makefile.in 2024-09-05T02:03:51.605 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/process.c 2024-09-05T02:03:51.607 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/globals.h 2024-09-05T02:03:51.609 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/commenter.c 2024-09-05T02:03:51.611 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/rewriter.c 2024-09-05T02:03:51.613 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/reader.c 2024-09-05T02:03:51.616 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/writer.c 2024-09-05T02:03:51.618 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/src/helpers.c 2024-09-05T02:03:51.620 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/NEWS 2024-09-05T02:03:51.622 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/compile 2024-09-05T02:03:51.624 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/depcomp 2024-09-05T02:03:51.626 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/aclocal.m4 2024-09-05T02:03:51.629 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/README 2024-09-05T02:03:51.631 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/configure 2024-09-05T02:03:51.640 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/configure.ac 2024-09-05T02:03:51.642 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/config.guess 2024-09-05T02:03:51.645 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/install-sh 2024-09-05T02:03:51.647 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/config.sub 2024-09-05T02:03:51.650 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/missing 2024-09-05T02:03:51.652 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/mkinstalldirs 2024-09-05T02:03:51.654 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/Makefile.am 2024-09-05T02:03:51.656 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/Makefile.in 2024-09-05T02:03:51.658 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/config.h.in 2024-09-05T02:03:51.660 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/AUTHORS 2024-09-05T02:03:51.662 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/INSTALL 2024-09-05T02:03:51.664 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/ChangeLog 2024-09-05T02:03:51.666 INFO:tasks.workunit.client.0.smithi103.stdout:blogbench-1.0/COPYING 2024-09-05T02:03:51.669 INFO:tasks.workunit.client.0.smithi103.stderr:+ cd blogbench-1.0/ 2024-09-05T02:03:51.670 INFO:tasks.workunit.client.0.smithi103.stdout:making blogbench 2024-09-05T02:03:51.671 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo 'making blogbench' 2024-09-05T02:03:51.671 INFO:tasks.workunit.client.0.smithi103.stderr:+ ./configure 2024-09-05T02:03:51.773 INFO:tasks.workunit.client.0.smithi103.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-05T02:03:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:52 smithi167 ceph-mon[35122]: pgmap v393: 129 pgs: 129 active+clean; 676 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-05T02:03:52.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:52 smithi139 ceph-mon[34262]: pgmap v393: 129 pgs: 129 active+clean; 676 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-05T02:03:52.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:52 smithi103 ceph-mon[30902]: pgmap v393: 129 pgs: 129 active+clean; 676 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-05T02:03:52.783 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether build environment is sane... yes 2024-09-05T02:03:52.798 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gawk... gawk 2024-09-05T02:03:52.819 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether make sets $(MAKE)... yes 2024-09-05T02:03:52.834 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc... gcc 2024-09-05T02:03:52.904 INFO:tasks.workunit.client.0.smithi103.stdout:checking for C compiler default output file name... a.out 2024-09-05T02:03:52.906 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether the C compiler works... yes 2024-09-05T02:03:52.910 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are cross compiling... no 2024-09-05T02:03:52.959 INFO:tasks.workunit.client.0.smithi103.stdout:checking for suffix of executables... 2024-09-05T02:03:53.005 INFO:tasks.workunit.client.0.smithi103.stdout:checking for suffix of object files... o 2024-09-05T02:03:53.047 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are using the GNU C compiler... yes 2024-09-05T02:03:53.091 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether gcc accepts -g... yes 2024-09-05T02:03:53.158 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-05T02:03:53.228 INFO:tasks.workunit.client.0.smithi103.stdout:checking for style of include used by make... GNU 2024-09-05T02:03:53.371 INFO:tasks.workunit.client.0.smithi103.stdout:checking dependency style of gcc... gcc3 2024-09-05T02:03:53.371 INFO:tasks.workunit.client.0.smithi103.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-05T02:03:53.445 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to run the C preprocessor... gcc -E 2024-09-05T02:03:53.505 INFO:tasks.workunit.client.0.smithi103.stdout:checking for egrep... grep -E 2024-09-05T02:03:53.519 INFO:tasks.workunit.client.0.smithi103.stdout:checking for AIX... no 2024-09-05T02:03:53.609 INFO:tasks.workunit.client.0.smithi103.stdout:checking for library containing strerror... none required 2024-09-05T02:03:53.795 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ANSI C header files... yes 2024-09-05T02:03:53.848 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-05T02:03:53.907 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/types.h... yes 2024-09-05T02:03:53.972 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/stat.h... yes 2024-09-05T02:03:54.037 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdlib.h... yes 2024-09-05T02:03:54.103 INFO:tasks.workunit.client.0.smithi103.stdout:checking for string.h... yes 2024-09-05T02:03:54.173 INFO:tasks.workunit.client.0.smithi103.stdout:checking for memory.h... yes 2024-09-05T02:03:54.238 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strings.h... yes 2024-09-05T02:03:54.303 INFO:tasks.workunit.client.0.smithi103.stdout:checking for inttypes.h... yes 2024-09-05T02:03:54.370 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdint.h... yes 2024-09-05T02:03:54.440 INFO:tasks.workunit.client.0.smithi103.stdout:checking for unistd.h... yes 2024-09-05T02:03:54.457 INFO:tasks.workunit.client.0.smithi103.stdout:checking for memory.h... (cached) yes 2024-09-05T02:03:54.467 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strings.h... (cached) yes 2024-09-05T02:03:54.481 INFO:tasks.workunit.client.0.smithi103.stdout:checking for unistd.h... (cached) yes 2024-09-05T02:03:54.554 INFO:tasks.workunit.client.0.smithi103.stdout:checking locale.h usability... yes 2024-09-05T02:03:54.581 INFO:tasks.workunit.client.0.smithi103.stdout:checking locale.h presence... yes 2024-09-05T02:03:54.582 INFO:tasks.workunit.client.0.smithi103.stdout:checking for locale.h... yes 2024-09-05T02:03:54.605 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdlib.h... (cached) yes 2024-09-05T02:03:54.615 INFO:tasks.workunit.client.0.smithi103.stdout:checking for string.h... (cached) yes 2024-09-05T02:03:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:54 smithi167 ceph-mon[35122]: pgmap v394: 129 pgs: 129 active+clean; 730 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-05T02:03:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:54 smithi139 ceph-mon[34262]: pgmap v394: 129 pgs: 129 active+clean; 730 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-05T02:03:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:54 smithi103 ceph-mon[30902]: pgmap v394: 129 pgs: 129 active+clean; 730 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-05T02:03:54.681 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/socket.h usability... yes 2024-09-05T02:03:54.711 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/socket.h presence... yes 2024-09-05T02:03:54.712 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/socket.h... yes 2024-09-05T02:03:54.790 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/time.h usability... yes 2024-09-05T02:03:54.818 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/time.h presence... yes 2024-09-05T02:03:54.819 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/time.h... yes 2024-09-05T02:03:54.894 INFO:tasks.workunit.client.0.smithi103.stdout:checking fcntl.h usability... yes 2024-09-05T02:03:54.922 INFO:tasks.workunit.client.0.smithi103.stdout:checking fcntl.h presence... yes 2024-09-05T02:03:54.923 INFO:tasks.workunit.client.0.smithi103.stdout:checking for fcntl.h... yes 2024-09-05T02:03:55.003 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/fcntl.h usability... yes 2024-09-05T02:03:55.033 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/fcntl.h presence... yes 2024-09-05T02:03:55.034 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/fcntl.h... yes 2024-09-05T02:03:55.143 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/param.h usability... yes 2024-09-05T02:03:55.173 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/param.h presence... yes 2024-09-05T02:03:55.174 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/param.h... yes 2024-09-05T02:03:55.239 INFO:tasks.workunit.client.0.smithi103.stdout:checking ioctl.h usability... no 2024-09-05T02:03:55.268 INFO:tasks.workunit.client.0.smithi103.stdout:checking ioctl.h presence... no 2024-09-05T02:03:55.269 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ioctl.h... no 2024-09-05T02:03:55.334 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/ioctl.h usability... yes 2024-09-05T02:03:55.361 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/ioctl.h presence... yes 2024-09-05T02:03:55.362 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/ioctl.h... yes 2024-09-05T02:03:55.430 INFO:tasks.workunit.client.0.smithi103.stdout:checking return type of signal handlers... void 2024-09-05T02:03:55.505 INFO:tasks.workunit.client.0.smithi103.stdout:checking for size_t... yes 2024-09-05T02:03:55.559 INFO:tasks.workunit.client.0.smithi103.stdout:checking for dev_t... yes 2024-09-05T02:03:55.619 INFO:tasks.workunit.client.0.smithi103.stdout:checking for short... yes 2024-09-05T02:03:55.722 INFO:tasks.workunit.client.0.smithi103.stdout:checking size of short... 2 2024-09-05T02:03:55.782 INFO:tasks.workunit.client.0.smithi103.stdout:checking for int... yes 2024-09-05T02:03:55.871 INFO:tasks.workunit.client.0.smithi103.stdout:checking size of int... 4 2024-09-05T02:03:55.930 INFO:tasks.workunit.client.0.smithi103.stdout:checking for long... yes 2024-09-05T02:03:56.027 INFO:tasks.workunit.client.0.smithi103.stdout:checking size of long... 8 2024-09-05T02:03:56.083 INFO:tasks.workunit.client.0.smithi103.stdout:checking for long long... yes 2024-09-05T02:03:56.174 INFO:tasks.workunit.client.0.smithi103.stdout:checking size of long long... 8 2024-09-05T02:03:56.234 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ssize_t... yes 2024-09-05T02:03:56.287 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sig_atomic_t... yes 2024-09-05T02:03:56.344 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether gcc needs -traditional... no 2024-09-05T02:03:56.391 INFO:tasks.workunit.client.0.smithi103.stdout:checking for an ANSI C-conforming const... yes 2024-09-05T02:03:56.427 INFO:tasks.workunit.client.0.smithi103.stdout:checking for inline... inline 2024-09-05T02:03:56.532 INFO:tasks.workunit.client.0.smithi103.stdout:checking for working alloca.h... yes 2024-09-05T02:03:56.599 INFO:tasks.workunit.client.0.smithi103.stdout:checking for alloca... yes 2024-09-05T02:03:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:56 smithi167 ceph-mon[35122]: pgmap v395: 129 pgs: 129 active+clean; 2.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 137 KiB/s wr, 26 op/s 2024-09-05T02:03:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:56 smithi139 ceph-mon[34262]: pgmap v395: 129 pgs: 129 active+clean; 2.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 137 KiB/s wr, 26 op/s 2024-09-05T02:03:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:56 smithi103 ceph-mon[30902]: pgmap v395: 129 pgs: 129 active+clean; 2.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 137 KiB/s wr, 26 op/s 2024-09-05T02:03:56.683 INFO:tasks.workunit.client.0.smithi103.stdout:checking for working memcmp... yes 2024-09-05T02:03:56.763 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strftime... yes 2024-09-05T02:03:56.859 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-05T02:03:56.942 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether stat accepts an empty string... no 2024-09-05T02:03:57.021 INFO:tasks.workunit.client.0.smithi103.stdout:checking for vprintf... yes 2024-09-05T02:03:57.100 INFO:tasks.workunit.client.0.smithi103.stdout:checking for _doprnt... no 2024-09-05T02:03:57.129 INFO:tasks.workunit.client.0.smithi103.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-05T02:03:57.143 INFO:tasks.workunit.client.0.smithi103.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-05T02:03:57.206 INFO:tasks.workunit.client.0.smithi103.stdout:checking for the pthreads library -lpthreads... no 2024-09-05T02:03:57.279 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether pthreads work without any flags... yes 2024-09-05T02:03:57.343 INFO:tasks.workunit.client.0.smithi103.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-05T02:03:57.344 INFO:tasks.workunit.client.0.smithi103.stdout:checking if more special flags are required for pthreads... no 2024-09-05T02:03:57.344 INFO:tasks.workunit.client.0.smithi103.stdout:checking for cc_r... gcc 2024-09-05T02:03:57.444 INFO:tasks.workunit.client.0.smithi103.stdout:checking for setrlimit... yes 2024-09-05T02:03:57.543 INFO:tasks.workunit.client.0.smithi103.stdout:checking for setproctitle... no 2024-09-05T02:03:57.627 INFO:tasks.workunit.client.0.smithi103.stdout:checking for getopt_long... yes 2024-09-05T02:03:57.714 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strtoull... yes 2024-09-05T02:03:57.797 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strtoq... yes 2024-09-05T02:03:57.884 INFO:tasks.workunit.client.0.smithi103.stdout:checking for snprintf... yes 2024-09-05T02:03:57.971 INFO:tasks.workunit.client.0.smithi103.stdout:checking for vsnprintf... yes 2024-09-05T02:03:58.058 INFO:tasks.workunit.client.0.smithi103.stdout:checking for vfprintf... yes 2024-09-05T02:03:58.143 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gethostname... yes 2024-09-05T02:03:58.227 INFO:tasks.workunit.client.0.smithi103.stdout:checking for setlocale... yes 2024-09-05T02:03:58.315 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sync... yes 2024-09-05T02:03:58.405 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether snprintf is C99 conformant... done 2024-09-05T02:03:58.421 INFO:tasks.workunit.client.0.smithi103.stdout:configure: creating ./config.status 2024-09-05T02:03:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:03:58 smithi167 ceph-mon[35122]: pgmap v396: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 154 KiB/s wr, 35 op/s 2024-09-05T02:03:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:03:58 smithi139 ceph-mon[34262]: pgmap v396: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 154 KiB/s wr, 35 op/s 2024-09-05T02:03:58.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:03:58 smithi103 ceph-mon[30902]: pgmap v396: 129 pgs: 129 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 154 KiB/s wr, 35 op/s 2024-09-05T02:03:58.662 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating Makefile 2024-09-05T02:03:58.684 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating src/Makefile 2024-09-05T02:03:58.711 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating man/Makefile 2024-09-05T02:03:58.734 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating config.h 2024-09-05T02:03:58.835 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: executing depfiles commands 2024-09-05T02:03:59.012 INFO:tasks.workunit.client.0.smithi103.stderr:+ make 2024-09-05T02:03:59.026 INFO:tasks.workunit.client.0.smithi103.stdout:make all-recursive 2024-09-05T02:03:59.032 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-05T02:03:59.036 INFO:tasks.workunit.client.0.smithi103.stdout:Making all in src 2024-09-05T02:03:59.059 INFO:tasks.workunit.client.0.smithi103.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-05T02:03:59.059 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-05T02:03:59.059 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-05T02:03:59.161 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-05T02:03:59.161 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-05T02:03:59.325 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-05T02:03:59.325 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-05T02:03:59.470 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-05T02:03:59.471 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-05T02:03:59.625 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-05T02:03:59.625 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-05T02:03:59.763 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-05T02:03:59.763 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-05T02:03:59.904 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-05T02:03:59.904 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-05T02:04:00.067 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-05T02:04:00.067 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-05T02:04:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.152+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3245988989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 msgr2=0x7fe4981031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.152+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3245988989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 0x7fe4981031d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009900 tx=0x7fe48c02a410 comp rx=0 tx=0).stop 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.152+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3245988989 shutdown_connections 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.152+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3245988989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe498103710 0x7fe498109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.152+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3245988989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 0x7fe4981031d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.152+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3245988989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498101b60 0x7fe498101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.152+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3245988989 >> 172.21.15.103:0/3245988989 conn(0x7fe4980fd2e0 msgr2=0x7fe4980ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.152+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3245988989 shutdown_connections 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.152+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3245988989 wait complete. 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe49cbd8640 1 Processor -- start 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe49cbd8640 1 -- start start 2024-09-05T02:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe49cbd8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498101b60 0x7fe498197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe49cbd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 0x7fe498198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe49cbd8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe498103710 0x7fe49819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe49cbd8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe49810d690 con 0x7fe498102d50 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe49cbd8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe49810d510 con 0x7fe498103710 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe49cbd8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe49810d810 con 0x7fe498101b60 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe495d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 0x7fe498198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe495d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 0x7fe498198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44330/0 (socket says 172.21.15.103:44330) 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe495d9b640 1 -- 172.21.15.103:0/3453280837 learned_addr learned my addr 172.21.15.103:0/3453280837 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe496d9d640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe498103710 0x7fe49819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe49659c640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498101b60 0x7fe498197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe495d9b640 1 -- 172.21.15.103:0/3453280837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498101b60 msgr2=0x7fe498197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe495d9b640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498101b60 0x7fe498197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe495d9b640 1 -- 172.21.15.103:0/3453280837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe498103710 msgr2=0x7fe49819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe495d9b640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe498103710 0x7fe49819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.153+0000 7fe495d9b640 1 -- 172.21.15.103:0/3453280837 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49819f960 con 0x7fe498102d50 2024-09-05T02:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.154+0000 7fe495d9b640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 0x7fe498198280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe48c005940 tx=0x7fe48c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.154+0000 7fe4877fe640 1 -- 172.21.15.103:0/3453280837 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c03f030 con 0x7fe498102d50 2024-09-05T02:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.154+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3453280837 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49819fbf0 con 0x7fe498102d50 2024-09-05T02:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.154+0000 7fe4877fe640 1 -- 172.21.15.103:0/3453280837 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe48c003cb0 con 0x7fe498102d50 2024-09-05T02:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.154+0000 7fe4877fe640 1 -- 172.21.15.103:0/3453280837 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c035070 con 0x7fe498102d50 2024-09-05T02:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.154+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3453280837 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4981a00e0 con 0x7fe498102d50 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe4877fe640 1 -- 172.21.15.103:0/3453280837 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe48c042030 con 0x7fe498102d50 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe49fa4b740 1 Processor -- start 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe49fa4b740 1 -- start start 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347f991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f9bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b347ef32f0 con 0x55b347f99780 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b347e9b270 con 0x55b347f93db0 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b347eef2b0 con 0x55b347f946f0 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe496d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.157+0000 7fe495d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f9bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe496d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f941b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:35626/0 (socket says 172.21.15.103:35626) 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe496d9d640 1 -- 172.21.15.103:0/1270439146 learned_addr learned my addr 172.21.15.103:0/1270439146 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe49659c640 1 --2- 172.21.15.103:0/1270439146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347f991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe496d9d640 1 -- 172.21.15.103:0/1270439146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 msgr2=0x55b347f991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe496d9d640 1 --2- 172.21.15.103:0/1270439146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347f991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe496d9d640 1 -- 172.21.15.103:0/1270439146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 msgr2=0x55b347f9bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe496d9d640 1 --2- 172.21.15.103:0/1270439146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f9bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe496d9d640 1 -- 172.21.15.103:0/1270439146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b347e9a340 con 0x55b347f93db0 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe4877fe640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4800a2c30 0x7fe4800a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe496d9d640 1 --2- 172.21.15.103:0/1270439146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f941b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe490009fd0 tx=0x7fe4900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22668807693f8d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe49659c640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4800a2c30 0x7fe4800a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe4877fe640 1 -- 172.21.15.103:0/3453280837 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe48c0ef900 con 0x7fe498102d50 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe484ff9640 1 -- 172.21.15.103:0/1270439146 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490004d20 con 0x55b347f93db0 2024-09-05T02:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe484ff9640 1 -- 172.21.15.103:0/1270439146 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe490004050 con 0x55b347f93db0 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.158+0000 7fe484ff9640 1 -- 172.21.15.103:0/1270439146 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490005260 con 0x55b347f93db0 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49659c640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4800a2c30 0x7fe4800a50f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55b347f95120 tx=0x7fe48800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49fa4b740 1 -- 172.21.15.103:0/1270439146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 msgr2=0x55b347f941b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/1270439146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f941b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe490009fd0 tx=0x7fe4900118d0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49fa4b740 1 -- 172.21.15.103:0/1270439146 shutdown_connections 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/1270439146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f9bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/1270439146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347f991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/1270439146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f941b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49fa4b740 1 -- 172.21.15.103:0/1270439146 >> 172.21.15.103:0/1270439146 conn(0x55b347f8b010 msgr2=0x55b347f8b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49fa4b740 1 -- 172.21.15.103:0/1270439146 shutdown_connections 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.159+0000 7fe49fa4b740 1 -- 172.21.15.103:0/1270439146 wait complete. 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49fa4b740 1 -- start start 2024-09-05T02:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49fa4b740 1 Processor -- start 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f93db0 0x55b347f9e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f946f0 0x55b347f9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347fa5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b347f9c750 con 0x55b347f99780 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b347e7c290 con 0x55b347f946f0 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b347f9c8d0 con 0x55b347f93db0 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49fa4b740 10 client.0 _ll_register_callbacks cb 0x55b347ead140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe496d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347fa5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe496d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347fa5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44350/0 (socket says 172.21.15.103:44350) 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe496d9d640 1 -- 172.21.15.103:0/2253056186 learned_addr learned my addr 172.21.15.103:0/2253056186 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe495d9b640 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f946f0 0x55b347f9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe49659c640 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f93db0 0x55b347f9e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe496d9d640 1 -- 172.21.15.103:0/2253056186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f93db0 msgr2=0x55b347f9e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe496d9d640 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f93db0 0x55b347f9e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe496d9d640 1 -- 172.21.15.103:0/2253056186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f946f0 msgr2=0x55b347f9e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe496d9d640 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f946f0 0x55b347f9e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe496d9d640 1 -- 172.21.15.103:0/2253056186 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b347c06210 con 0x55b347f99780 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.160+0000 7fe495d9b640 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f946f0 0x55b347f9e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe496d9d640 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347fa5540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe49001b040 tx=0x7fe490004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ab4c5353948faf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe47dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe47dffb640 1 -- 172.21.15.103:0/2253056186 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490028020 con 0x55b347f99780 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe47dffb640 1 -- 172.21.15.103:0/2253056186 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe490003c40 con 0x55b347f99780 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2253056186 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe490004050 con 0x55b347f99780 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe47dffb640 1 -- 172.21.15.103:0/2253056186 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe490033050 con 0x55b347f99780 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe49fa4b740 10 client.15609 fetch_fsmap learned FSMap version 29 2024-09-05T02:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2253056186 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b347cd2620 con 0x55b347f99780 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe47dffb640 1 -- 172.21.15.103:0/2253056186 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe490004050 con 0x55b347f99780 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe47dffb640 1 -- 172.21.15.103:0/2253056186 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe49000f4e0 con 0x55b347f99780 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe49fa4b740 10 client.15609 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe49fa4b740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe49fa4b740 20 client.15609 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b347fa9b30 0x55b347fabf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe49fa4b740 4 client.15609 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.161+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2253056186 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b347f00a20 con 0x55b347fa9b30 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.162+0000 7fe49659c640 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b347fa9b30 0x55b347fabf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.162+0000 7fe49659c640 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b347fa9b30 0x55b347fabf20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be15225242aef384 server_cookie=4efe0de4568d492d in_seq=0 out_seq=0 2024-09-05T02:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.162+0000 7fe47dffb640 10 client.15609 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe47dffb640 1 -- 172.21.15.103:0/2253056186 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b347f00a20 con 0x55b347fa9b30 2024-09-05T02:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe47dffb640 10 client.15609 handle_command_reply: tid=0 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 client.15609 shutdown 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2253056186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 msgr2=0x55b347fa5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347fa5540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe49001b040 tx=0x7fe490004cf0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2253056186 shutdown_connections 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b347fa9b30 0x55b347fabf20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347fa5540 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f946f0 0x55b347f9e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2253056186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f93db0 0x55b347f9e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe47dffb640 0 client.15609 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2253056186 >> 172.21.15.103:0/2253056186 conn(0x55b347f8b010 msgr2=0x55b347f8c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2253056186 shutdown_connections 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2253056186 wait complete. 2024-09-05T02:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.163+0000 7fe49fa4b740 20 client.15609 trim_cache size 0 max 16384 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe49fa4b740 1 Processor -- start 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe49fa4b740 1 -- start start 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f8bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347fa5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f8c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981a84e0 con 0x55b347f99780 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981aa020 con 0x55b347f93db0 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981a8bb0 con 0x55b347f946f0 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe496d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f8c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe49659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f8bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe496d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f8c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44354/0 (socket says 172.21.15.103:44354) 2024-09-05T02:04:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe496d9d640 1 -- 172.21.15.103:0/3843973168 learned_addr learned my addr 172.21.15.103:0/3843973168 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe495d9b640 1 --2- 172.21.15.103:0/3843973168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347fa5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe496d9d640 1 -- 172.21.15.103:0/3843973168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 msgr2=0x55b347fa5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe496d9d640 1 --2- 172.21.15.103:0/3843973168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347fa5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe496d9d640 1 -- 172.21.15.103:0/3843973168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 msgr2=0x55b347f8bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe496d9d640 1 --2- 172.21.15.103:0/3843973168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f8bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.180+0000 7fe496d9d640 1 -- 172.21.15.103:0/3843973168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b347fac500 con 0x55b347f99780 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.181+0000 7fe496d9d640 1 --2- 172.21.15.103:0/3843973168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f8c400 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe490011910 tx=0x7fe490005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97cdbd9029b69c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.181+0000 7fe47ffff640 1 -- 172.21.15.103:0/3843973168 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490028040 con 0x55b347f99780 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.181+0000 7fe47ffff640 1 -- 172.21.15.103:0/3843973168 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe490011dd0 con 0x55b347f99780 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.181+0000 7fe47ffff640 1 -- 172.21.15.103:0/3843973168 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490019430 con 0x55b347f99780 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 -- 172.21.15.103:0/3843973168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 msgr2=0x55b347f8c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/3843973168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f8c400 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe490011910 tx=0x7fe490005f10 comp rx=0 tx=0).stop 2024-09-05T02:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 -- 172.21.15.103:0/3843973168 shutdown_connections 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/3843973168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f99780 0x55b347f8c400 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/3843973168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347fa5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/3843973168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f93db0 0x55b347f8bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 -- 172.21.15.103:0/3843973168 >> 172.21.15.103:0/3843973168 conn(0x55b347f8b010 msgr2=0x55b347f9e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 -- 172.21.15.103:0/3843973168 shutdown_connections 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 -- 172.21.15.103:0/3843973168 wait complete. 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 -- start start 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 Processor -- start 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f93db0 0x55b347fa27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347fa2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f99780 0x55b347fa3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981a9950 con 0x55b347f93db0 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981a9280 con 0x55b347f99780 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981a64c0 con 0x55b347f946f0 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49fa4b740 10 client.0 _ll_register_callbacks cb 0x7fe48c0f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.182+0000 7fe49659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f93db0 0x55b347fa27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe496d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f99780 0x55b347fa3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe49659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f93db0 0x55b347fa27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44362/0 (socket says 172.21.15.103:44362) 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe49659c640 1 -- 172.21.15.103:0/2251505021 learned_addr learned my addr 172.21.15.103:0/2251505021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe495d9b640 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347fa2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe496d9d640 1 -- 172.21.15.103:0/2251505021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 msgr2=0x55b347fa2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe496d9d640 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347fa2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe496d9d640 1 -- 172.21.15.103:0/2251505021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f93db0 msgr2=0x55b347fa27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe496d9d640 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f93db0 0x55b347fa27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe496d9d640 1 -- 172.21.15.103:0/2251505021 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b347fa4340 con 0x55b347f99780 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe49659c640 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f93db0 0x55b347fa27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe496d9d640 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f99780 0x55b347fa3d30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe49000ffe0 tx=0x7fe490004940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4403c9d6996267de server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe47dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe47dffb640 1 -- 172.21.15.103:0/2251505021 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490045060 con 0x55b347f99780 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2251505021 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe490011dd0 con 0x55b347f99780 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe47dffb640 1 -- 172.21.15.103:0/2251505021 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe490003e40 con 0x55b347f99780 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe47dffb640 1 -- 172.21.15.103:0/2251505021 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe49004c050 con 0x55b347f99780 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe49fa4b740 10 client.25085 fetch_fsmap learned FSMap version 29 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.183+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2251505021 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b347fa5650 con 0x55b347f99780 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe47dffb640 1 -- 172.21.15.103:0/2251505021 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe490011dd0 con 0x55b347f99780 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe47dffb640 1 -- 172.21.15.103:0/2251505021 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe490040b20 con 0x55b347f99780 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe49fa4b740 10 client.25085 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T02:04:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe49fa4b740 10 client.25085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe49fa4b740 20 client.25085 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b348014cf0 0x55b348017110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe49fa4b740 4 client.25085 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:04:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2251505021 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b347efad70 con 0x55b348014cf0 2024-09-05T02:04:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe49659c640 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b348014cf0 0x55b348017110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe47dffb640 10 client.25085 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.184+0000 7fe49659c640 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b348014cf0 0x55b348017110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8910b380aab0fcc8 server_cookie=4626fa6e396bfc5a in_seq=0 out_seq=0 2024-09-05T02:04:00.215 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-05T02:04:00.216 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe47dffb640 1 -- 172.21.15.103:0/2251505021 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b347efad70 con 0x55b348014cf0 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe47dffb640 10 client.25085 handle_command_reply: tid=0 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 client.25085 shutdown 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2251505021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f99780 msgr2=0x55b347fa3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f99780 0x55b347fa3d30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe49000ffe0 tx=0x7fe490004940 comp rx=0 tx=0).stop 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2251505021 shutdown_connections 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b348014cf0 0x55b348017110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b347f99780 0x55b347fa3d30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b347f946f0 0x55b347fa2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe47dffb640 0 client.25085 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 --2- 172.21.15.103:0/2251505021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b347f93db0 0x55b347fa27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2251505021 >> 172.21.15.103:0/2251505021 conn(0x55b347f8b010 msgr2=0x55b347f899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2251505021 shutdown_connections 2024-09-05T02:04:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 1 -- 172.21.15.103:0/2251505021 wait complete. 2024-09-05T02:04:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.235+0000 7fe49fa4b740 20 client.25085 trim_cache size 0 max 16384 2024-09-05T02:04:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:04:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:04:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:04:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4800a2c30 msgr2=0x7fe4800a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4800a2c30 0x7fe4800a50f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55b347f95120 tx=0x7fe48800a2d0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 msgr2=0x7fe498198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 0x7fe498198280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe48c005940 tx=0x7fe48c0042f0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3453280837 shutdown_connections 2024-09-05T02:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe498103710 0x7fe49819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe498102d50 0x7fe498198280 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe4800a2c30 0x7fe4800a50f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 --2- 172.21.15.103:0/3453280837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498101b60 0x7fe498197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3453280837 >> 172.21.15.103:0/3453280837 conn(0x7fe4980fd2e0 msgr2=0x7fe4981061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3453280837 shutdown_connections 2024-09-05T02:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:00.236+0000 7fe49cbd8640 1 -- 172.21.15.103:0/3453280837 wait complete. 2024-09-05T02:04:00.350 INFO:tasks.workunit.client.0.smithi103.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-05T02:04:00.351 INFO:tasks.workunit.client.0.smithi103.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-05T02:04:00.442 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-05T02:04:00.516 INFO:tasks.workunit.client.0.smithi103.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-05T02:04:00.520 INFO:tasks.workunit.client.0.smithi103.stdout:Making all in man 2024-09-05T02:04:00.532 INFO:tasks.workunit.client.0.smithi103.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-05T02:04:00.532 INFO:tasks.workunit.client.0.smithi103.stdout:make[2]: Nothing to be done for 'all'. 2024-09-05T02:04:00.532 INFO:tasks.workunit.client.0.smithi103.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-05T02:04:00.540 INFO:tasks.workunit.client.0.smithi103.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-05T02:04:00.542 INFO:tasks.workunit.client.0.smithi103.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-05T02:04:00.542 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-05T02:04:00.543 INFO:tasks.workunit.client.0.smithi103.stderr:+ cd src 2024-09-05T02:04:00.545 INFO:tasks.workunit.client.0.smithi103.stderr:+ mkdir blogtest_in 2024-09-05T02:04:00.548 INFO:tasks.workunit.client.0.smithi103.stdout:running blogbench 2024-09-05T02:04:00.548 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo 'running blogbench' 2024-09-05T02:04:00.549 INFO:tasks.workunit.client.0.smithi103.stderr:+ ./blogbench -d blogtest_in 2024-09-05T02:04:00.549 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-09-05T02:04:00.549 INFO:tasks.workunit.client.0.smithi103.stdout:Frequency = 10 secs 2024-09-05T02:04:00.549 INFO:tasks.workunit.client.0.smithi103.stdout:Scratch dir = [blogtest_in] 2024-09-05T02:04:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:00 smithi167 ceph-mon[35122]: pgmap v397: 129 pgs: 129 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 416 KiB/s wr, 64 op/s 2024-09-05T02:04:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:00 smithi139 ceph-mon[34262]: pgmap v397: 129 pgs: 129 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 416 KiB/s wr, 64 op/s 2024-09-05T02:04:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:00 smithi103 ceph-mon[30902]: pgmap v397: 129 pgs: 129 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 416 KiB/s wr, 64 op/s 2024-09-05T02:04:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:02 smithi167 ceph-mon[35122]: pgmap v398: 129 pgs: 129 active+clean; 5.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 484 KiB/s wr, 117 op/s 2024-09-05T02:04:02.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:02 smithi139 ceph-mon[34262]: pgmap v398: 129 pgs: 129 active+clean; 5.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 484 KiB/s wr, 117 op/s 2024-09-05T02:04:02.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:02 smithi103 ceph-mon[30902]: pgmap v398: 129 pgs: 129 active+clean; 5.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 484 KiB/s wr, 117 op/s 2024-09-05T02:04:03.871 INFO:tasks.workunit.client.0.smithi103.stdout:Spawning 3 writers... 2024-09-05T02:04:03.872 INFO:tasks.workunit.client.0.smithi103.stdout:Spawning 1 rewriters... 2024-09-05T02:04:03.872 INFO:tasks.workunit.client.0.smithi103.stdout:Spawning 5 commenters... 2024-09-05T02:04:03.872 INFO:tasks.workunit.client.0.smithi103.stdout:Spawning 100 readers... 2024-09-05T02:04:03.873 INFO:tasks.workunit.client.0.smithi103.stdout:Benchmarking for 30 iterations. 2024-09-05T02:04:03.873 INFO:tasks.workunit.client.0.smithi103.stdout:The test will run during 5 minutes. 2024-09-05T02:04:03.873 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-09-05T02:04:03.873 INFO:tasks.workunit.client.0.smithi103.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-05T02:04:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:04 smithi167 ceph-mon[35122]: pgmap v399: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 497 KiB/s wr, 134 op/s 2024-09-05T02:04:04.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:04 smithi139 ceph-mon[34262]: pgmap v399: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 497 KiB/s wr, 134 op/s 2024-09-05T02:04:04.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:04 smithi103 ceph-mon[30902]: pgmap v399: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 497 KiB/s wr, 134 op/s 2024-09-05T02:04:05.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:05 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:05.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:05 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:05.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:05 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:06 smithi167 ceph-mon[35122]: pgmap v400: 129 pgs: 129 active+clean; 6.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 607 KiB/s wr, 168 op/s 2024-09-05T02:04:06.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:06 smithi139 ceph-mon[34262]: pgmap v400: 129 pgs: 129 active+clean; 6.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 607 KiB/s wr, 168 op/s 2024-09-05T02:04:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:06 smithi103 ceph-mon[30902]: pgmap v400: 129 pgs: 129 active+clean; 6.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 607 KiB/s wr, 168 op/s 2024-09-05T02:04:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:08 smithi167 ceph-mon[35122]: pgmap v401: 129 pgs: 129 active+clean; 9.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 765 KiB/s wr, 168 op/s 2024-09-05T02:04:08.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:08 smithi139 ceph-mon[34262]: pgmap v401: 129 pgs: 129 active+clean; 9.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 765 KiB/s wr, 168 op/s 2024-09-05T02:04:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:08 smithi103 ceph-mon[30902]: pgmap v401: 129 pgs: 129 active+clean; 9.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 765 KiB/s wr, 168 op/s 2024-09-05T02:04:10.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:10 smithi167 ceph-mon[35122]: pgmap v402: 129 pgs: 129 active+clean; 12 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 977 KiB/s wr, 176 op/s 2024-09-05T02:04:10.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:10 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:10.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:10 smithi139 ceph-mon[34262]: pgmap v402: 129 pgs: 129 active+clean; 12 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 977 KiB/s wr, 176 op/s 2024-09-05T02:04:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:10 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:10 smithi103 ceph-mon[30902]: pgmap v402: 129 pgs: 129 active+clean; 12 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 977 KiB/s wr, 176 op/s 2024-09-05T02:04:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:10 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:12 smithi167 ceph-mon[35122]: pgmap v403: 129 pgs: 129 active+clean; 28 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-09-05T02:04:12.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:12 smithi139 ceph-mon[34262]: pgmap v403: 129 pgs: 129 active+clean; 28 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-09-05T02:04:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:12 smithi103 ceph-mon[30902]: pgmap v403: 129 pgs: 129 active+clean; 28 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-09-05T02:04:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.375+0000 7f5a4570a640 1 -- 172.21.15.103:0/842504213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a40101f60 msgr2=0x7f5a40109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.375+0000 7f5a4570a640 1 --2- 172.21.15.103:0/842504213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a40101f60 0x7f5a40109dd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ca0 tx=0x7f5a3802a8c0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.375+0000 7f5a4570a640 1 -- 172.21.15.103:0/842504213 shutdown_connections 2024-09-05T02:04:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.375+0000 7f5a4570a640 1 --2- 172.21.15.103:0/842504213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a40101f60 0x7f5a40109dd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.375+0000 7f5a4570a640 1 --2- 172.21.15.103:0/842504213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a401015a0 0x7f5a40101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.375+0000 7f5a4570a640 1 --2- 172.21.15.103:0/842504213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a40103070 0x7f5a40103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.375+0000 7f5a4570a640 1 -- 172.21.15.103:0/842504213 >> 172.21.15.103:0/842504213 conn(0x7f5a400fd0f0 msgr2=0x7f5a400ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.375+0000 7f5a4570a640 1 -- 172.21.15.103:0/842504213 shutdown_connections 2024-09-05T02:04:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.375+0000 7f5a4570a640 1 -- 172.21.15.103:0/842504213 wait complete. 2024-09-05T02:04:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a4570a640 1 Processor -- start 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a4570a640 1 -- start start 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a4570a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a401015a0 0x7f5a40197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a4570a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a40101f60 0x7f5a401981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a4570a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a40103070 0x7f5a4019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a4570a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010d5c0 con 0x7f5a40101f60 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a4570a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010d440 con 0x7f5a401015a0 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a4570a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010d740 con 0x7f5a40103070 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a3effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a401015a0 0x7f5a40197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a3effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a401015a0 0x7f5a40197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:60582/0 (socket says 172.21.15.103:60582) 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a3effd640 1 -- 172.21.15.103:0/3728167754 learned_addr learned my addr 172.21.15.103:0/3728167754 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a40101f60 0x7f5a401981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.376+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a40103070 0x7f5a4019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a3effd640 1 -- 172.21.15.103:0/3728167754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a40103070 msgr2=0x7f5a4019f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a3effd640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a40103070 0x7f5a4019f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a3effd640 1 -- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a40101f60 msgr2=0x7f5a401981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a3effd640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a40101f60 0x7f5a401981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a3effd640 1 -- 172.21.15.103:0/3728167754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4019f8f0 con 0x7f5a401015a0 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a40103070 0x7f5a4019f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a40101f60 0x7f5a401981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a3effd640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a401015a0 0x7f5a40197cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9c0 tx=0x7f5a3000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a27fff640 1 -- 172.21.15.103:0/3728167754 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000cd80 con 0x7f5a401015a0 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a4570a640 1 -- 172.21.15.103:0/3728167754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4019fbe0 con 0x7f5a401015a0 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a27fff640 1 -- 172.21.15.103:0/3728167754 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a300035f0 con 0x7f5a401015a0 2024-09-05T02:04:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a27fff640 1 -- 172.21.15.103:0/3728167754 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30010690 con 0x7f5a401015a0 2024-09-05T02:04:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.377+0000 7f5a4570a640 1 -- 172.21.15.103:0/3728167754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a400746b0 con 0x7f5a401015a0 2024-09-05T02:04:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a484a2740 1 Processor -- start 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a484a2740 1 -- start start 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0650cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 0x557f06511f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06512540 0x557f06514a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a484a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557f064702f0 con 0x557f0650cb70 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a484a2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557f06418270 con 0x557f0650d4b0 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a484a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557f0646c2b0 con 0x557f06512540 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a27fff640 1 -- 172.21.15.103:0/3728167754 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5a300037a0 con 0x7f5a401015a0 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0650cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0650cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39834/0 (socket says 172.21.15.103:39834) 2024-09-05T02:04:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/1998116276 learned_addr learned my addr 172.21.15.103:0/1998116276 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.381+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/1998116276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06512540 0x557f06514a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a3effd640 1 --2- 172.21.15.103:0/1998116276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 0x557f06511f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/1998116276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06512540 msgr2=0x557f06514a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/1998116276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06512540 0x557f06514a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/1998116276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 msgr2=0x557f06511f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/1998116276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 0x557f06511f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/1998116276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f06417340 con 0x557f0650cb70 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/1998116276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0650cf70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5a3802ae30 tx=0x7f5a38009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5c1d101382fbd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a27fff640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5a280a2bc0 0x7f5a280a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a257fa640 1 -- 172.21.15.103:0/1998116276 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38031650 con 0x557f0650cb70 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a257fa640 1 -- 172.21.15.103:0/1998116276 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a38036070 con 0x557f0650cb70 2024-09-05T02:04:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.382+0000 7f5a257fa640 1 -- 172.21.15.103:0/1998116276 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a380043a0 con 0x557f0650cb70 2024-09-05T02:04:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.383+0000 7f5a27fff640 1 -- 172.21.15.103:0/3728167754 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5a300cf980 con 0x7f5a401015a0 2024-09-05T02:04:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.383+0000 7f5a3effd640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5a280a2bc0 0x7f5a280a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.383+0000 7f5a484a2740 1 -- 172.21.15.103:0/1998116276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 msgr2=0x557f0650cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.383+0000 7f5a3effd640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5a280a2bc0 0x7f5a280a5080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5a300cd440 tx=0x7f5a30097a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.383+0000 7f5a484a2740 1 --2- 172.21.15.103:0/1998116276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0650cf70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5a3802ae30 tx=0x7f5a38009a20 comp rx=0 tx=0).stop 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.384+0000 7f5a484a2740 1 -- 172.21.15.103:0/1998116276 shutdown_connections 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.384+0000 7f5a484a2740 1 --2- 172.21.15.103:0/1998116276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06512540 0x557f06514a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.384+0000 7f5a484a2740 1 --2- 172.21.15.103:0/1998116276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 0x557f06511f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.384+0000 7f5a484a2740 1 --2- 172.21.15.103:0/1998116276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0650cf70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.384+0000 7f5a484a2740 1 -- 172.21.15.103:0/1998116276 >> 172.21.15.103:0/1998116276 conn(0x557f065081e0 msgr2=0x557f065085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.384+0000 7f5a484a2740 1 -- 172.21.15.103:0/1998116276 shutdown_connections 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.384+0000 7f5a484a2740 1 -- 172.21.15.103:0/1998116276 wait complete. 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.384+0000 7f5a484a2740 1 -- start start 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.384+0000 7f5a484a2740 1 Processor -- start 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f06516d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 0x557f06517260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 0x557f0651e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a484a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557f065154b0 con 0x557f0650cb70 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a484a2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557f063f9290 con 0x557f06512540 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a484a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557f06515630 con 0x557f0650d4b0 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a484a2740 10 client.0 _ll_register_callbacks cb 0x557f0642a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 0x557f0651e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 0x557f0651e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:60594/0 (socket says 172.21.15.103:60594) 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/4148176978 learned_addr learned my addr 172.21.15.103:0/4148176978 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 0x557f06517260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.385+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f06516d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a3e7fc640 1 -- 172.21.15.103:0/4148176978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 msgr2=0x557f0651e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 0x557f0651e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a3e7fc640 1 -- 172.21.15.103:0/4148176978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 msgr2=0x557f06516d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f06516d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a3e7fc640 1 -- 172.21.15.103:0/4148176978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f06183210 con 0x557f0650d4b0 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f06516d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 0x557f06517260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a401991b0 tx=0x7f5a3400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7694b17a5dcebeca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a22ff5640 1 -- 172.21.15.103:0/4148176978 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34012110 con 0x557f0650d4b0 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a22ff5640 1 -- 172.21.15.103:0/4148176978 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5a34004050 con 0x557f0650d4b0 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a484a2740 1 -- 172.21.15.103:0/4148176978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a38036070 con 0x557f0650d4b0 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a22ff5640 1 -- 172.21.15.103:0/4148176978 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a34025050 con 0x557f0650d4b0 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a484a2740 10 client.25066 fetch_fsmap learned FSMap version 31 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.386+0000 7f5a484a2740 1 -- 172.21.15.103:0/4148176978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557f0624f620 con 0x557f0650d4b0 2024-09-05T02:04:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.387+0000 7f5a22ff5640 1 -- 172.21.15.103:0/4148176978 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a38036070 con 0x557f0650d4b0 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.387+0000 7f5a22ff5640 1 -- 172.21.15.103:0/4148176978 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f5a34010430 con 0x557f0650d4b0 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.387+0000 7f5a484a2740 10 client.25066 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.387+0000 7f5a484a2740 10 client.25066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.387+0000 7f5a484a2740 20 client.25066 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.387+0000 7f5a484a2740 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557f06522ad0 0x557f06524ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.387+0000 7f5a484a2740 4 client.25066 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.387+0000 7f5a484a2740 1 -- 172.21.15.103:0/4148176978 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f0647da20 con 0x557f06522ad0 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.387+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557f06522ad0 0x557f06524ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.388+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557f06522ad0 0x557f06524ec0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f5b70873b05fa9d server_cookie=225f6f38c580b663 in_seq=0 out_seq=0 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.388+0000 7f5a22ff5640 10 client.25066 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.388+0000 7f5a22ff5640 1 -- 172.21.15.103:0/4148176978 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f0647da20 con 0x557f06522ad0 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.388+0000 7f5a22ff5640 10 client.25066 handle_command_reply: tid=0 2024-09-05T02:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.388+0000 7f5a484a2740 1 client.25066 shutdown 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.388+0000 7f5a484a2740 1 -- 172.21.15.103:0/4148176978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 msgr2=0x557f06517260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.388+0000 7f5a484a2740 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 0x557f06517260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a401991b0 tx=0x7f5a3400f790 comp rx=0 tx=0).stop 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.388+0000 7f5a484a2740 1 -- 172.21.15.103:0/4148176978 shutdown_connections 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.388+0000 7f5a484a2740 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557f06522ad0 0x557f06524ec0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.389+0000 7f5a484a2740 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 0x557f0651e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.389+0000 7f5a484a2740 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 0x557f06517260 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.389+0000 7f5a484a2740 1 --2- 172.21.15.103:0/4148176978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f06516d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.389+0000 7f5a22ff5640 0 client.25066 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.389+0000 7f5a484a2740 1 -- 172.21.15.103:0/4148176978 >> 172.21.15.103:0/4148176978 conn(0x557f065081e0 msgr2=0x557f065099f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.389+0000 7f5a484a2740 1 -- 172.21.15.103:0/4148176978 shutdown_connections 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.389+0000 7f5a484a2740 1 -- 172.21.15.103:0/4148176978 wait complete. 2024-09-05T02:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.389+0000 7f5a484a2740 20 client.25066 trim_cache size 0 max 16384 2024-09-05T02:04:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.412+0000 7f5a484a2740 1 Processor -- start 2024-09-05T02:04:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.412+0000 7f5a484a2740 1 -- start start 2024-09-05T02:04:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.412+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f065071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.412+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 0x557f0651e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f065071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f065071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39862/0 (socket says 172.21.15.103:39862) 2024-09-05T02:04:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/1968051505 learned_addr learned my addr 172.21.15.103:0/1968051505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a484a2740 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06512540 0x557f065076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 0x557f0651e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a484a2740 1 -- 172.21.15.103:0/1968051505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ac7b0 con 0x557f0650cb70 2024-09-05T02:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a484a2740 1 -- 172.21.15.103:0/1968051505 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ae3e0 con 0x557f0650d4b0 2024-09-05T02:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a484a2740 1 -- 172.21.15.103:0/1968051505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ace80 con 0x557f06512540 2024-09-05T02:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3e7fc640 1 -- 172.21.15.103:0/1968051505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06512540 msgr2=0x557f065076e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T02:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06512540 0x557f065076e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3e7fc640 1 -- 172.21.15.103:0/1968051505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 msgr2=0x557f065071a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f065071a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3e7fc640 1 -- 172.21.15.103:0/1968051505 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f06525230 con 0x557f0650d4b0 2024-09-05T02:04:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.413+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f065071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.414+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 0x557f0651e260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a340109f0 tx=0x7f5a340123a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f233fc1b1d0c7b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.414+0000 7f5a24ff9640 1 -- 172.21.15.103:0/1968051505 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3402e070 con 0x557f0650d4b0 2024-09-05T02:04:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.414+0000 7f5a24ff9640 1 -- 172.21.15.103:0/1968051505 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a34005560 con 0x557f0650d4b0 2024-09-05T02:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.414+0000 7f5a24ff9640 1 -- 172.21.15.103:0/1968051505 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34003850 con 0x557f0650d4b0 2024-09-05T02:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.415+0000 7f5a484a2740 1 -- 172.21.15.103:0/1968051505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 msgr2=0x557f0651e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.415+0000 7f5a484a2740 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 0x557f0651e260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a340109f0 tx=0x7f5a340123a0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.415+0000 7f5a484a2740 1 -- 172.21.15.103:0/1968051505 shutdown_connections 2024-09-05T02:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.415+0000 7f5a484a2740 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06512540 0x557f065076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.415+0000 7f5a484a2740 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f0650d4b0 0x557f0651e260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.415+0000 7f5a484a2740 1 --2- 172.21.15.103:0/1968051505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f065071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.415+0000 7f5a484a2740 1 -- 172.21.15.103:0/1968051505 >> 172.21.15.103:0/1968051505 conn(0x557f065081e0 msgr2=0x557f06506a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.416+0000 7f5a484a2740 1 -- 172.21.15.103:0/1968051505 shutdown_connections 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.416+0000 7f5a484a2740 1 -- 172.21.15.103:0/1968051505 wait complete. 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.416+0000 7f5a484a2740 1 -- start start 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.416+0000 7f5a484a2740 1 Processor -- start 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.417+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0651b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.417+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 0x557f0651baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.417+0000 7f5a484a2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 0x557f0651cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.417+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0651b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.417+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0651b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39870/0 (socket says 172.21.15.103:39870) 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.417+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/473061666 learned_addr learned my addr 172.21.15.103:0/473061666 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.417+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401adc50 con 0x557f0650cb70 2024-09-05T02:04:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.417+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 0x557f0651baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.418+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ad550 con 0x557f06512540 2024-09-05T02:04:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.418+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401aa790 con 0x557f0650d4b0 2024-09-05T02:04:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.418+0000 7f5a484a2740 10 client.0 _ll_register_callbacks cb 0x7f5a34009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.418+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/473061666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 msgr2=0x557f0651baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.418+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 0x557f0651baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.419+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/473061666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 msgr2=0x557f0651cb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.419+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 0x557f0651cb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.419+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/473061666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0651d110 con 0x557f0650cb70 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.419+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0651b5b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ca0 tx=0x7f5a3802adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e5deeda98c49b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.419+0000 7f5a22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.419+0000 7f5a22ff5640 1 -- 172.21.15.103:0/473061666 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38049040 con 0x557f0650cb70 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.419+0000 7f5a22ff5640 1 -- 172.21.15.103:0/473061666 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5a38031b40 con 0x557f0650cb70 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.419+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a34005560 con 0x557f0650cb70 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a22ff5640 1 -- 172.21.15.103:0/473061666 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a38031d40 con 0x557f0650cb70 2024-09-05T02:04:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a484a2740 10 client.15633 fetch_fsmap learned FSMap version 31 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557f0651e4b0 con 0x557f0650cb70 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a22ff5640 1 -- 172.21.15.103:0/473061666 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a38056070 con 0x557f0650cb70 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a22ff5640 1 -- 172.21.15.103:0/473061666 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f5a38030d10 con 0x557f0650cb70 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a484a2740 10 client.15633 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a484a2740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a484a2740 20 client.15633 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a484a2740 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557f06522ad0 0x557f06598730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a484a2740 4 client.15633 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.420+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5a34005560 con 0x557f06522ad0 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.421+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557f06522ad0 0x557f06598730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.421+0000 7f5a22ff5640 10 client.15633 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.422+0000 7f5a3e7fc640 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557f06522ad0 0x557f06598730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4806e9a85d334f server_cookie=6db35000f7c75c0f in_seq=0 out_seq=0 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.435+0000 7f5a22ff5640 1 -- 172.21.15.103:0/473061666 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5a34005560 con 0x557f06522ad0 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.435+0000 7f5a22ff5640 10 client.15633 handle_command_reply: tid=0 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.435+0000 7f5a484a2740 1 client.15633 shutdown 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.435+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 msgr2=0x557f0651b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.435+0000 7f5a484a2740 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0651b5b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ca0 tx=0x7f5a3802adc0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 shutdown_connections 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a484a2740 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557f06522ad0 0x557f06598730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a484a2740 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f06512540 0x557f0651cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a484a2740 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0650d4b0 0x557f0651baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a484a2740 1 --2- 172.21.15.103:0/473061666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f0650cb70 0x557f0651b5b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 >> 172.21.15.103:0/473061666 conn(0x557f065081e0 msgr2=0x557f065072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a22ff5640 0 client.15633 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 shutdown_connections 2024-09-05T02:04:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a484a2740 1 -- 172.21.15.103:0/473061666 wait complete. 2024-09-05T02:04:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.436+0000 7f5a484a2740 20 client.15633 trim_cache size 0 max 16384 2024-09-05T02:04:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:04:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:04:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:04:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:04:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 -- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5a280a2bc0 msgr2=0x7f5a280a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5a280a2bc0 0x7f5a280a5080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5a300cd440 tx=0x7f5a30097a00 comp rx=0 tx=0).stop 2024-09-05T02:04:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 -- 172.21.15.103:0/3728167754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a401015a0 msgr2=0x7f5a40197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a401015a0 0x7f5a40197cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9c0 tx=0x7f5a3000ee80 comp rx=0 tx=0).stop 2024-09-05T02:04:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 -- 172.21.15.103:0/3728167754 shutdown_connections 2024-09-05T02:04:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f5a280a2bc0 0x7f5a280a5080 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a40103070 0x7f5a4019f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a40101f60 0x7f5a401981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 --2- 172.21.15.103:0/3728167754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a401015a0 0x7f5a40197cb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 -- 172.21.15.103:0/3728167754 >> 172.21.15.103:0/3728167754 conn(0x7f5a400fd0f0 msgr2=0x7f5a40105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 -- 172.21.15.103:0/3728167754 shutdown_connections 2024-09-05T02:04:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:13.437+0000 7f5a4570a640 1 -- 172.21.15.103:0/3728167754 wait complete. 2024-09-05T02:04:13.873 INFO:tasks.workunit.client.0.smithi103.stdout: 7 13280 520 3335 381 3321 798 2024-09-05T02:04:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:14 smithi167 ceph-mon[35122]: pgmap v404: 129 pgs: 129 active+clean; 30 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-09-05T02:04:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:14 smithi139 ceph-mon[34262]: pgmap v404: 129 pgs: 129 active+clean; 30 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-09-05T02:04:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:14 smithi103 ceph-mon[30902]: pgmap v404: 129 pgs: 129 active+clean; 30 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-09-05T02:04:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:04:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:15 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:04:15.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:15 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:04:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:15.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:15 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:16 smithi167 ceph-mon[35122]: pgmap v405: 129 pgs: 129 active+clean; 33 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-09-05T02:04:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:04:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:04:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:16 smithi139 ceph-mon[34262]: pgmap v405: 129 pgs: 129 active+clean; 33 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-09-05T02:04:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:04:16.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:04:16.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:16 smithi103 ceph-mon[30902]: pgmap v405: 129 pgs: 129 active+clean; 33 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-09-05T02:04:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:04:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:04:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:04:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:18 smithi167 ceph-mon[35122]: pgmap v406: 129 pgs: 129 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 215 op/s 2024-09-05T02:04:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:18 smithi139 ceph-mon[34262]: pgmap v406: 129 pgs: 129 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 215 op/s 2024-09-05T02:04:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:18 smithi103 ceph-mon[30902]: pgmap v406: 129 pgs: 129 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 215 op/s 2024-09-05T02:04:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:20 smithi167 ceph-mon[35122]: pgmap v407: 129 pgs: 129 active+clean; 43 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 218 op/s 2024-09-05T02:04:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:20 smithi139 ceph-mon[34262]: pgmap v407: 129 pgs: 129 active+clean; 43 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 218 op/s 2024-09-05T02:04:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:20 smithi103 ceph-mon[30902]: pgmap v407: 129 pgs: 129 active+clean; 43 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 218 op/s 2024-09-05T02:04:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:22 smithi167 ceph-mon[35122]: pgmap v408: 129 pgs: 129 active+clean; 48 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 233 op/s 2024-09-05T02:04:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:22 smithi139 ceph-mon[34262]: pgmap v408: 129 pgs: 129 active+clean; 48 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 233 op/s 2024-09-05T02:04:22.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:22 smithi103 ceph-mon[30902]: pgmap v408: 129 pgs: 129 active+clean; 48 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 233 op/s 2024-09-05T02:04:23.873 INFO:tasks.workunit.client.0.smithi103.stdout: 13 15015 239 5891 238 6234 289 2024-09-05T02:04:24.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:24 smithi167 ceph-mon[35122]: pgmap v409: 129 pgs: 129 active+clean; 49 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-05T02:04:24.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:24 smithi139 ceph-mon[34262]: pgmap v409: 129 pgs: 129 active+clean; 49 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-05T02:04:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:24 smithi103 ceph-mon[30902]: pgmap v409: 129 pgs: 129 active+clean; 49 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-05T02:04:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.564+0000 7f9572b33640 1 -- 172.21.15.103:0/1813823183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 msgr2=0x7f956c10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.564+0000 7f9572b33640 1 --2- 172.21.15.103:0/1813823183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 0x7f956c10c9b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9560009900 tx=0x7f956002a410 comp rx=0 tx=0).stop 2024-09-05T02:04:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.564+0000 7f9572b33640 1 -- 172.21.15.103:0/1813823183 shutdown_connections 2024-09-05T02:04:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.564+0000 7f9572b33640 1 --2- 172.21.15.103:0/1813823183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f956c10e100 0x7f956c1149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.564+0000 7f9572b33640 1 --2- 172.21.15.103:0/1813823183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f956c10d760 0x7f956c10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.564+0000 7f9572b33640 1 --2- 172.21.15.103:0/1813823183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 0x7f956c10c9b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.564+0000 7f9572b33640 1 -- 172.21.15.103:0/1813823183 >> 172.21.15.103:0/1813823183 conn(0x7f956c06c290 msgr2=0x7f956c06c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.564+0000 7f9572b33640 1 -- 172.21.15.103:0/1813823183 shutdown_connections 2024-09-05T02:04:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.564+0000 7f9572b33640 1 -- 172.21.15.103:0/1813823183 wait complete. 2024-09-05T02:04:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.565+0000 7f9572b33640 1 Processor -- start 2024-09-05T02:04:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.565+0000 7f9572b33640 1 -- start start 2024-09-05T02:04:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f9572b33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 0x7f956c1a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f9572b33640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f956c10d760 0x7f956c1a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f9572b33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f956c10e100 0x7f956c1ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f9572b33640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f956c06ea40 con 0x7f956c10c5b0 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f9572b33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f956c06e8c0 con 0x7f956c10e100 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f95708cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 0x7f956c1a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f95708cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 0x7f956c1a6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33270/0 (socket says 172.21.15.103:33270) 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f95708cf640 1 -- 172.21.15.103:0/1755587442 learned_addr learned my addr 172.21.15.103:0/1755587442 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f9572b33640 1 -- 172.21.15.103:0/1755587442 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f956c06ebc0 con 0x7f956c10d760 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f956bfff640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f956c10d760 0x7f956c1a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f95710d0640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f956c10e100 0x7f956c1ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f956bfff640 1 -- 172.21.15.103:0/1755587442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f956c10e100 msgr2=0x7f956c1ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f956bfff640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f956c10e100 0x7f956c1ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f956bfff640 1 -- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 msgr2=0x7f956c1a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f956bfff640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 0x7f956c1a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f956bfff640 1 -- 172.21.15.103:0/1755587442 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c1ae790 con 0x7f956c10d760 2024-09-05T02:04:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.566+0000 7f95708cf640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 0x7f956c1a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.567+0000 7f956bfff640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f956c10d760 0x7f956c1a70b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f955800d6d0 tx=0x7f955800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.567+0000 7f9569ffb640 1 -- 172.21.15.103:0/1755587442 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558017070 con 0x7f956c10d760 2024-09-05T02:04:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.567+0000 7f9569ffb640 1 -- 172.21.15.103:0/1755587442 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9558009820 con 0x7f956c10d760 2024-09-05T02:04:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.567+0000 7f9572b33640 1 -- 172.21.15.103:0/1755587442 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956c1aea80 con 0x7f956c10d760 2024-09-05T02:04:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.567+0000 7f9572b33640 1 -- 172.21.15.103:0/1755587442 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956c1af020 con 0x7f956c10d760 2024-09-05T02:04:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.567+0000 7f9569ffb640 1 -- 172.21.15.103:0/1755587442 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558004c40 con 0x7f956c10d760 2024-09-05T02:04:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.569+0000 7f9569ffb640 1 -- 172.21.15.103:0/1755587442 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f95580099d0 con 0x7f956c10d760 2024-09-05T02:04:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.570+0000 7f95758c9740 1 Processor -- start 2024-09-05T02:04:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.570+0000 7f95758c9740 1 -- start start 2024-09-05T02:04:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.570+0000 7f95758c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95710d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95710d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33284/0 (socket says 172.21.15.103:33284) 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95758c9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95758c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95758c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a6c9052f0 con 0x560a6c9a19a0 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95758c9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a6c8ad270 con 0x560a6c9a22e0 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95758c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560a6c9012b0 con 0x560a6c9a7370 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95710d0640 1 -- 172.21.15.103:0/1793804458 learned_addr learned my addr 172.21.15.103:0/1793804458 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f956bfff640 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95708cf640 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95710d0640 1 -- 172.21.15.103:0/1793804458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 msgr2=0x560a6c9a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95710d0640 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95710d0640 1 -- 172.21.15.103:0/1793804458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 msgr2=0x560a6c9a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95710d0640 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95710d0640 1 -- 172.21.15.103:0/1793804458 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6c8ac340 con 0x560a6c9a19a0 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.571+0000 7f95708cf640 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.572+0000 7f9569ffb640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f955c0a2cb0 0x7f955c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.572+0000 7f95708cf640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f955c0a2cb0 0x7f955c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.572+0000 7f95710d0640 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9a1da0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f956400e380 tx=0x7f956400e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d105897c410c9ff server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.572+0000 7f9569ffb640 1 -- 172.21.15.103:0/1755587442 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f95580ceea0 con 0x7f956c10d760 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.572+0000 7f9556ffd640 1 -- 172.21.15.103:0/1793804458 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564014070 con 0x560a6c9a19a0 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.572+0000 7f9556ffd640 1 -- 172.21.15.103:0/1793804458 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9564003b70 con 0x560a6c9a19a0 2024-09-05T02:04:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.572+0000 7f9556ffd640 1 -- 172.21.15.103:0/1793804458 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564010800 con 0x560a6c9a19a0 2024-09-05T02:04:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95758c9740 1 -- 172.21.15.103:0/1793804458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 msgr2=0x560a6c9a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95758c9740 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9a1da0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f956400e380 tx=0x7f956400e840 comp rx=0 tx=0).stop 2024-09-05T02:04:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95708cf640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f955c0a2cb0 0x7f955c0a5170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f95600314e0 tx=0x7f95600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95758c9740 1 -- 172.21.15.103:0/1793804458 shutdown_connections 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95758c9740 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95758c9740 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95758c9740 1 --2- 172.21.15.103:0/1793804458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9a1da0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95758c9740 1 -- 172.21.15.103:0/1793804458 >> 172.21.15.103:0/1793804458 conn(0x560a6c99d010 msgr2=0x560a6c99d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95758c9740 1 -- 172.21.15.103:0/1793804458 shutdown_connections 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.573+0000 7f95758c9740 1 -- 172.21.15.103:0/1793804458 wait complete. 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95758c9740 1 -- start start 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95758c9740 1 Processor -- start 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95758c9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a19a0 0x560a6c9abc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95758c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a22e0 0x560a6c9ac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95758c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95758c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a6c9aa340 con 0x560a6c9a22e0 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95758c9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a6c88e290 con 0x560a6c9a19a0 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95758c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560a6c9aa4c0 con 0x560a6c9a7370 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95758c9740 10 client.0 _ll_register_callbacks cb 0x560a6c8bf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95708cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a19a0 0x560a6c9abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95708cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a19a0 0x560a6c9abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:50818/0 (socket says 172.21.15.103:50818) 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95708cf640 1 -- 172.21.15.103:0/3573306155 learned_addr learned my addr 172.21.15.103:0/3573306155 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f956bfff640 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a22e0 0x560a6c9ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.574+0000 7f95710d0640 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f95708cf640 1 -- 172.21.15.103:0/3573306155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 msgr2=0x560a6c9b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f95708cf640 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f95708cf640 1 -- 172.21.15.103:0/3573306155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a22e0 msgr2=0x560a6c9ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f95708cf640 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a22e0 0x560a6c9ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f95708cf640 1 -- 172.21.15.103:0/3573306155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6c618210 con 0x560a6c9a19a0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f956bfff640 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a22e0 0x560a6c9ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f95708cf640 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a19a0 0x560a6c9abc20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c00 tx=0x7f9560009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0bd3c948cf191 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f954bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f954bfff640 1 -- 172.21.15.103:0/3573306155 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956004c070 con 0x560a6c9a19a0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f95758c9740 1 -- 172.21.15.103:0/3573306155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9564003b70 con 0x560a6c9a19a0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f954bfff640 1 -- 172.21.15.103:0/3573306155 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9560003ac0 con 0x560a6c9a19a0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f954bfff640 1 -- 172.21.15.103:0/3573306155 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9560042070 con 0x560a6c9a19a0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.575+0000 7f95758c9740 10 client.25112 fetch_fsmap learned FSMap version 31 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f95758c9740 1 -- 172.21.15.103:0/3573306155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560a6c6e4620 con 0x560a6c9a19a0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f954bfff640 1 -- 172.21.15.103:0/3573306155 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9564003b70 con 0x560a6c9a19a0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f954bfff640 1 -- 172.21.15.103:0/3573306155 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f956003c440 con 0x560a6c9a19a0 2024-09-05T02:04:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f95758c9740 10 client.25112 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f95758c9740 10 client.25112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f95758c9740 20 client.25112 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f95758c9740 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a6c9b7910 0x560a6c9b9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f95758c9740 4 client.25112 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f95758c9740 1 -- 172.21.15.103:0/3573306155 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a6c912a20 con 0x560a6c9b7910 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.576+0000 7f956bfff640 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a6c9b7910 0x560a6c9b9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.577+0000 7f954bfff640 10 client.25112 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.577+0000 7f956bfff640 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a6c9b7910 0x560a6c9b9d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e9e6ab5ca8e4c7 server_cookie=4c259377bd541bbf in_seq=0 out_seq=0 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.577+0000 7f954bfff640 1 -- 172.21.15.103:0/3573306155 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a6c912a20 con 0x560a6c9b7910 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.577+0000 7f954bfff640 10 client.25112 handle_command_reply: tid=0 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.577+0000 7f95758c9740 1 client.25112 shutdown 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 -- 172.21.15.103:0/3573306155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a19a0 msgr2=0x560a6c9abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a19a0 0x560a6c9abc20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c00 tx=0x7f9560009260 comp rx=0 tx=0).stop 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 -- 172.21.15.103:0/3573306155 shutdown_connections 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a6c9b7910 0x560a6c9b9d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a22e0 0x560a6c9ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 --2- 172.21.15.103:0/3573306155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a19a0 0x560a6c9abc20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f954bfff640 0 client.25112 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 -- 172.21.15.103:0/3573306155 >> 172.21.15.103:0/3573306155 conn(0x560a6c99d010 msgr2=0x560a6c99e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 -- 172.21.15.103:0/3573306155 shutdown_connections 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 1 -- 172.21.15.103:0/3573306155 wait complete. 2024-09-05T02:04:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.578+0000 7f95758c9740 20 client.25112 trim_cache size 0 max 16384 2024-09-05T02:04:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.596+0000 7f95758c9740 1 Processor -- start 2024-09-05T02:04:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.596+0000 7f95758c9740 1 -- start start 2024-09-05T02:04:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.596+0000 7f95758c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.596+0000 7f95758c9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9ab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95708cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95708cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33296/0 (socket says 172.21.15.103:33296) 2024-09-05T02:04:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95708cf640 1 -- 172.21.15.103:0/989099156 learned_addr learned my addr 172.21.15.103:0/989099156 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95758c9740 1 --2- 172.21.15.103:0/989099156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95758c9740 1 -- 172.21.15.103:0/989099156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f956c1b9540 con 0x560a6c9a19a0 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95758c9740 1 -- 172.21.15.103:0/989099156 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f956c1bb170 con 0x560a6c9a22e0 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95758c9740 1 -- 172.21.15.103:0/989099156 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f956c1b9c10 con 0x560a6c9a7370 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f956bfff640 1 --2- 172.21.15.103:0/989099156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95708cf640 1 -- 172.21.15.103:0/989099156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 msgr2=0x560a6c9b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95708cf640 1 --2- 172.21.15.103:0/989099156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95708cf640 1 -- 172.21.15.103:0/989099156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 msgr2=0x560a6c9ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95708cf640 1 --2- 172.21.15.103:0/989099156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95708cf640 1 -- 172.21.15.103:0/989099156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6c9ab760 con 0x560a6c9a19a0 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95708cf640 1 --2- 172.21.15.103:0/989099156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9ac040 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9560009230 tx=0x7f956002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a24f9ce5aa6c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95567fc640 1 -- 172.21.15.103:0/989099156 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560062020 con 0x560a6c9a19a0 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95567fc640 1 -- 172.21.15.103:0/989099156 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956003c440 con 0x560a6c9a19a0 2024-09-05T02:04:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.597+0000 7f95567fc640 1 -- 172.21.15.103:0/989099156 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956004c070 con 0x560a6c9a19a0 2024-09-05T02:04:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 -- 172.21.15.103:0/989099156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 msgr2=0x560a6c9ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 --2- 172.21.15.103:0/989099156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9ac040 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9560009230 tx=0x7f956002ab10 comp rx=0 tx=0).stop 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 -- 172.21.15.103:0/989099156 shutdown_connections 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 --2- 172.21.15.103:0/989099156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a7370 0x560a6c9b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 --2- 172.21.15.103:0/989099156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 --2- 172.21.15.103:0/989099156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a19a0 0x560a6c9ac040 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 -- 172.21.15.103:0/989099156 >> 172.21.15.103:0/989099156 conn(0x560a6c99d010 msgr2=0x560a6c99c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 -- 172.21.15.103:0/989099156 shutdown_connections 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 -- 172.21.15.103:0/989099156 wait complete. 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.598+0000 7f95758c9740 1 -- start start 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95758c9740 1 Processor -- start 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95758c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a19a0 0x560a6c9b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95758c9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95758c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a7370 0x560a6c9b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95758c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f956c1ba9e0 con 0x560a6c9a7370 2024-09-05T02:04:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95758c9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f956c1ba2e0 con 0x560a6c9a22e0 2024-09-05T02:04:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95758c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f956c1b7520 con 0x560a6c9a19a0 2024-09-05T02:04:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95758c9740 10 client.0 _ll_register_callbacks cb 0x7f9560005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95710d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a7370 0x560a6c9b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95710d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a7370 0x560a6c9b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33302/0 (socket says 172.21.15.103:33302) 2024-09-05T02:04:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95710d0640 1 -- 172.21.15.103:0/2869169071 learned_addr learned my addr 172.21.15.103:0/2869169071 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f956bfff640 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95708cf640 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a19a0 0x560a6c9b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95710d0640 1 -- 172.21.15.103:0/2869169071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a19a0 msgr2=0x560a6c9b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95710d0640 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a19a0 0x560a6c9b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95710d0640 1 -- 172.21.15.103:0/2869169071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 msgr2=0x560a6c9b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95710d0640 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95710d0640 1 -- 172.21.15.103:0/2869169071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6c9b2520 con 0x560a6c9a7370 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f956bfff640 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f95710d0640 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a7370 0x560a6c9b1f10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f956400e380 tx=0x7f9564004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f141359a1dfca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.599+0000 7f954bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f954bfff640 1 -- 172.21.15.103:0/2869169071 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564014070 con 0x560a6c9a7370 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f954bfff640 1 -- 172.21.15.103:0/2869169071 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9564010900 con 0x560a6c9a7370 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f95758c9740 1 -- 172.21.15.103:0/2869169071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f956003c440 con 0x560a6c9a7370 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f954bfff640 1 -- 172.21.15.103:0/2869169071 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f956402e050 con 0x560a6c9a7370 2024-09-05T02:04:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f95758c9740 10 client.15651 fetch_fsmap learned FSMap version 31 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f95758c9740 1 -- 172.21.15.103:0/2869169071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560a6ca22740 con 0x560a6c9a7370 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f954bfff640 1 -- 172.21.15.103:0/2869169071 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956003c440 con 0x560a6c9a7370 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f954bfff640 1 -- 172.21.15.103:0/2869169071 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f9564023430 con 0x560a6c9a7370 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f95758c9740 10 client.15651 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f95758c9740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f95758c9740 20 client.15651 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f95758c9740 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a6ca22e40 0x560a6ca25260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f95758c9740 4 client.15651 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.600+0000 7f95758c9740 1 -- 172.21.15.103:0/2869169071 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a6c90cd70 con 0x560a6ca22e40 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.601+0000 7f95708cf640 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a6ca22e40 0x560a6ca25260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.601+0000 7f954bfff640 10 client.15651 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.601+0000 7f95708cf640 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a6ca22e40 0x560a6ca25260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad0d8123ef36e4 server_cookie=805397215e81e1b6 in_seq=0 out_seq=0 2024-09-05T02:04:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:26 smithi167 ceph-mon[35122]: pgmap v410: 129 pgs: 129 active+clean; 55 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-05T02:04:26.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:26 smithi139 ceph-mon[34262]: pgmap v410: 129 pgs: 129 active+clean; 55 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-05T02:04:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:26 smithi103 ceph-mon[30902]: pgmap v410: 129 pgs: 129 active+clean; 55 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-05T02:04:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.694+0000 7f954bfff640 1 -- 172.21.15.103:0/2869169071 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a6c90cd70 con 0x560a6ca22e40 2024-09-05T02:04:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.694+0000 7f954bfff640 10 client.15651 handle_command_reply: tid=0 2024-09-05T02:04:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.694+0000 7f95758c9740 1 client.15651 shutdown 2024-09-05T02:04:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 -- 172.21.15.103:0/2869169071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a7370 msgr2=0x560a6c9b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a7370 0x560a6c9b1f10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f956400e380 tx=0x7f9564004f90 comp rx=0 tx=0).stop 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 -- 172.21.15.103:0/2869169071 shutdown_connections 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a6ca22e40 0x560a6ca25260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a6c9a7370 0x560a6c9b1f10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a6c9a22e0 0x560a6c9b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 --2- 172.21.15.103:0/2869169071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a6c9a19a0 0x560a6c9b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f954bfff640 0 client.15651 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 -- 172.21.15.103:0/2869169071 >> 172.21.15.103:0/2869169071 conn(0x560a6c99d010 msgr2=0x560a6c99d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 -- 172.21.15.103:0/2869169071 shutdown_connections 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 1 -- 172.21.15.103:0/2869169071 wait complete. 2024-09-05T02:04:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.695+0000 7f95758c9740 20 client.15651 trim_cache size 0 max 16384 2024-09-05T02:04:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:04:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:04:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:04:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:04:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 -- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f955c0a2cb0 msgr2=0x7f955c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f955c0a2cb0 0x7f955c0a5170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f95600314e0 tx=0x7f95600058d0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 -- 172.21.15.103:0/1755587442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f956c10d760 msgr2=0x7f956c1a70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f956c10d760 0x7f956c1a70b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f955800d6d0 tx=0x7f955800db90 comp rx=0 tx=0).stop 2024-09-05T02:04:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 -- 172.21.15.103:0/1755587442 shutdown_connections 2024-09-05T02:04:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f956c10e100 0x7f956c1ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f955c0a2cb0 0x7f955c0a5170 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f956c10d760 0x7f956c1a70b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 --2- 172.21.15.103:0/1755587442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f956c10c5b0 0x7f956c1a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 -- 172.21.15.103:0/1755587442 >> 172.21.15.103:0/1755587442 conn(0x7f956c06c290 msgr2=0x7f956c109670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 -- 172.21.15.103:0/1755587442 shutdown_connections 2024-09-05T02:04:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:26.696+0000 7f9572b33640 1 -- 172.21.15.103:0/1755587442 wait complete. 2024-09-05T02:04:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:28 smithi167 ceph-mon[35122]: pgmap v411: 129 pgs: 129 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-09-05T02:04:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:28 smithi139 ceph-mon[34262]: pgmap v411: 129 pgs: 129 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-09-05T02:04:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:28 smithi103 ceph-mon[30902]: pgmap v411: 129 pgs: 129 active+clean; 57 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-09-05T02:04:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:30 smithi167 ceph-mon[35122]: pgmap v412: 129 pgs: 129 active+clean; 59 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.8 MiB/s wr, 137 op/s 2024-09-05T02:04:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:30 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:30.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:30 smithi139 ceph-mon[34262]: pgmap v412: 129 pgs: 129 active+clean; 59 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.8 MiB/s wr, 137 op/s 2024-09-05T02:04:30.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:30 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:30 smithi103 ceph-mon[30902]: pgmap v412: 129 pgs: 129 active+clean; 59 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.8 MiB/s wr, 137 op/s 2024-09-05T02:04:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:30 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:32 smithi167 ceph-mon[35122]: pgmap v413: 129 pgs: 129 active+clean; 65 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-05T02:04:32.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:32 smithi139 ceph-mon[34262]: pgmap v413: 129 pgs: 129 active+clean; 65 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-05T02:04:32.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:32 smithi103 ceph-mon[30902]: pgmap v413: 129 pgs: 129 active+clean; 65 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-05T02:04:33.874 INFO:tasks.workunit.client.0.smithi103.stdout: 13 15009 50 6901 206 6665 144 2024-09-05T02:04:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:34 smithi167 ceph-mon[35122]: pgmap v414: 129 pgs: 129 active+clean; 66 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-09-05T02:04:34.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:34 smithi139 ceph-mon[34262]: pgmap v414: 129 pgs: 129 active+clean; 66 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-09-05T02:04:34.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:34 smithi103 ceph-mon[30902]: pgmap v414: 129 pgs: 129 active+clean; 66 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-09-05T02:04:35.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:35 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:35.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:35 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:35.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:35 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:04:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:36 smithi167 ceph-mon[35122]: pgmap v415: 129 pgs: 129 active+clean; 67 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-09-05T02:04:36.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:36 smithi139 ceph-mon[34262]: pgmap v415: 129 pgs: 129 active+clean; 67 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-09-05T02:04:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:36 smithi103 ceph-mon[30902]: pgmap v415: 129 pgs: 129 active+clean; 67 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-09-05T02:04:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:38 smithi167 ceph-mon[35122]: pgmap v416: 129 pgs: 129 active+clean; 69 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-09-05T02:04:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:38 smithi139 ceph-mon[34262]: pgmap v416: 129 pgs: 129 active+clean; 69 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-09-05T02:04:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:38 smithi103 ceph-mon[30902]: pgmap v416: 129 pgs: 129 active+clean; 69 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-09-05T02:04:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.809+0000 7f103e303640 1 -- 172.21.15.103:0/2472017215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1038100a40 msgr2=0x7f1038100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.809+0000 7f103e303640 1 --2- 172.21.15.103:0/2472017215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1038100a40 0x7f1038100e40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1028009900 tx=0x7f102802a410 comp rx=0 tx=0).stop 2024-09-05T02:04:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.809+0000 7f103e303640 1 -- 172.21.15.103:0/2472017215 shutdown_connections 2024-09-05T02:04:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.809+0000 7f103e303640 1 --2- 172.21.15.103:0/2472017215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10381025f0 0x7f1038108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.809+0000 7f103e303640 1 --2- 172.21.15.103:0/2472017215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1038101c30 0x7f10381020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.809+0000 7f103e303640 1 --2- 172.21.15.103:0/2472017215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1038100a40 0x7f1038100e40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.809+0000 7f103e303640 1 -- 172.21.15.103:0/2472017215 >> 172.21.15.103:0/2472017215 conn(0x7f10380fc1e0 msgr2=0x7f10380fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.809+0000 7f103e303640 1 -- 172.21.15.103:0/2472017215 shutdown_connections 2024-09-05T02:04:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.809+0000 7f103e303640 1 -- 172.21.15.103:0/2472017215 wait complete. 2024-09-05T02:04:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.810+0000 7f103e303640 1 Processor -- start 2024-09-05T02:04:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.810+0000 7f103e303640 1 -- start start 2024-09-05T02:04:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.810+0000 7f103e303640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1038101c30 0x7f103806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.810+0000 7f103e303640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10381025f0 0x7f103819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f103e303640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f103819d1b0 0x7f103819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f103e303640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f103810c5c0 con 0x7f103819d1b0 2024-09-05T02:04:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f103e303640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f103810c440 con 0x7f10381025f0 2024-09-05T02:04:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f103e303640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f103810c740 con 0x7f1038101c30 2024-09-05T02:04:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f103c8a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f103819d1b0 0x7f103819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f10377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10381025f0 0x7f103819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f103c8a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f103819d1b0 0x7f103819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48646/0 (socket says 172.21.15.103:48646) 2024-09-05T02:04:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f103c8a0640 1 -- 172.21.15.103:0/1884768870 learned_addr learned my addr 172.21.15.103:0/1884768870 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f1037fff640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1038101c30 0x7f103806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f10377fe640 1 -- 172.21.15.103:0/1884768870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1038101c30 msgr2=0x7f103806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f10377fe640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1038101c30 0x7f103806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f10377fe640 1 -- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f103819d1b0 msgr2=0x7f103819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f10377fe640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f103819d1b0 0x7f103819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f10377fe640 1 -- 172.21.15.103:0/1884768870 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103819ff80 con 0x7f10381025f0 2024-09-05T02:04:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f103c8a0640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f103819d1b0 0x7f103819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f10377fe640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10381025f0 0x7f103819cc70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e9f0 tx=0x7f102c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.811+0000 7f10357fa640 1 -- 172.21.15.103:0/1884768870 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c00ccc0 con 0x7f10381025f0 2024-09-05T02:04:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.812+0000 7f103e303640 1 -- 172.21.15.103:0/1884768870 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10381a0210 con 0x7f10381025f0 2024-09-05T02:04:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.812+0000 7f10357fa640 1 -- 172.21.15.103:0/1884768870 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f102c0108e0 con 0x7f10381025f0 2024-09-05T02:04:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.812+0000 7f10357fa640 1 -- 172.21.15.103:0/1884768870 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c018880 con 0x7f10381025f0 2024-09-05T02:04:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.812+0000 7f103e303640 1 -- 172.21.15.103:0/1884768870 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10381a0760 con 0x7f10381025f0 2024-09-05T02:04:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.814+0000 7f10357fa640 1 -- 172.21.15.103:0/1884768870 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f102c010a90 con 0x7f10381025f0 2024-09-05T02:04:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f1041098740 1 Processor -- start 2024-09-05T02:04:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f1041098740 1 -- start start 2024-09-05T02:04:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f1041098740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f1041098740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbd1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f1041098740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbd4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f1041098740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f5bb302f0 con 0x561f5bbd2370 2024-09-05T02:04:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f1041098740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561f5bad8270 con 0x561f5bbcc9a0 2024-09-05T02:04:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f1041098740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f5bb2c2b0 con 0x561f5bbcd2e0 2024-09-05T02:04:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f10377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbd4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f103c8a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f10357fa640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f10200a2c30 0x7f10200a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f103c8a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:51498/0 (socket says 172.21.15.103:51498) 2024-09-05T02:04:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f103c8a0640 1 -- 172.21.15.103:0/3644686126 learned_addr learned my addr 172.21.15.103:0/3644686126 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.815+0000 7f1037fff640 1 --2- 172.21.15.103:0/3644686126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbd1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f103c8a0640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f10200a2c30 0x7f10200a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f10357fa640 1 -- 172.21.15.103:0/1884768870 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f102c0cf7f0 con 0x7f10381025f0 2024-09-05T02:04:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f10377fe640 1 -- 172.21.15.103:0/3644686126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 msgr2=0x561f5bbd1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f10377fe640 1 --2- 172.21.15.103:0/3644686126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbd1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f10377fe640 1 -- 172.21.15.103:0/3644686126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 msgr2=0x561f5bbccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f10377fe640 1 --2- 172.21.15.103:0/3644686126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f10377fe640 1 -- 172.21.15.103:0/3644686126 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5bad7340 con 0x561f5bbd2370 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f1037fff640 1 --2- 172.21.15.103:0/3644686126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbd1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f10377fe640 1 --2- 172.21.15.103:0/3644686126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbd4830 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f102c0ccf90 tx=0x7f102c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc88dd33a8c4adad server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f103c8a0640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f10200a2c30 0x7f10200a50f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1030013fd0 tx=0x7f1030013560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f101e7fc640 1 -- 172.21.15.103:0/3644686126 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c026040 con 0x561f5bbd2370 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f101e7fc640 1 -- 172.21.15.103:0/3644686126 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f102c0a4910 con 0x561f5bbd2370 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.816+0000 7f101e7fc640 1 -- 172.21.15.103:0/3644686126 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c09c160 con 0x561f5bbd2370 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 -- 172.21.15.103:0/3644686126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 msgr2=0x561f5bbd4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 --2- 172.21.15.103:0/3644686126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbd4830 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f102c0ccf90 tx=0x7f102c010f40 comp rx=0 tx=0).stop 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 -- 172.21.15.103:0/3644686126 shutdown_connections 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 --2- 172.21.15.103:0/3644686126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbd4830 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 --2- 172.21.15.103:0/3644686126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbd1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 --2- 172.21.15.103:0/3644686126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 -- 172.21.15.103:0/3644686126 >> 172.21.15.103:0/3644686126 conn(0x561f5bbc8010 msgr2=0x561f5bbc8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 -- 172.21.15.103:0/3644686126 shutdown_connections 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 -- 172.21.15.103:0/3644686126 wait complete. 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.817+0000 7f1041098740 1 -- start start 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1041098740 1 Processor -- start 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1041098740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbd6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1041098740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbd70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1041098740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbde0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1041098740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f5bbd52e0 con 0x561f5bbd2370 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1041098740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561f5bab9290 con 0x561f5bbcc9a0 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1041098740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f5bbd5460 con 0x561f5bbcd2e0 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1037fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbd6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1037fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbd6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:51508/0 (socket says 172.21.15.103:51508) 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1037fff640 1 -- 172.21.15.103:0/4251897901 learned_addr learned my addr 172.21.15.103:0/4251897901 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f1041098740 10 client.0 _ll_register_callbacks cb 0x561f5baea140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.818+0000 7f103c8a0640 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbde0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f10377fe640 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbd70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f1037fff640 1 -- 172.21.15.103:0/4251897901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 msgr2=0x561f5bbd70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f1037fff640 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbd70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f1037fff640 1 -- 172.21.15.103:0/4251897901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 msgr2=0x561f5bbde0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f1037fff640 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbde0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f1037fff640 1 -- 172.21.15.103:0/4251897901 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5b843210 con 0x561f5bbcc9a0 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f103c8a0640 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbde0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f1037fff640 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbd6b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1028009230 tx=0x7f1028004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86290546c590ef09 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f10177fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f10177fe640 1 -- 172.21.15.103:0/4251897901 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028035070 con 0x561f5bbcc9a0 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f10177fe640 1 -- 172.21.15.103:0/4251897901 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1028005020 con 0x561f5bbcc9a0 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f1041098740 1 -- 172.21.15.103:0/4251897901 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f102c0a4910 con 0x561f5bbcc9a0 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f10177fe640 1 -- 172.21.15.103:0/4251897901 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1028051050 con 0x561f5bbcc9a0 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f1041098740 10 client.25130 fetch_fsmap learned FSMap version 33 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.819+0000 7f1041098740 1 -- 172.21.15.103:0/4251897901 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561f5b90f620 con 0x561f5bbcc9a0 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f10177fe640 1 -- 172.21.15.103:0/4251897901 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f102c0a4910 con 0x561f5bbcc9a0 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f10177fe640 1 -- 172.21.15.103:0/4251897901 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f1028003680 con 0x561f5bbcc9a0 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f1041098740 10 client.25130 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:04:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f1041098740 10 client.25130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f1041098740 20 client.25130 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f1041098740 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x561f5bbe0ec0 0x561f5bbe3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f1041098740 4 client.25130 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f1041098740 1 -- 172.21.15.103:0/4251897901 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f5bb3da20 con 0x561f5bbe0ec0 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f10377fe640 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x561f5bbe0ec0 0x561f5bbe3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f10177fe640 10 client.25130 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.820+0000 7f10377fe640 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x561f5bbe0ec0 0x561f5bbe3380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829520aa8ed50246 server_cookie=cc75fa11f914e16b in_seq=0 out_seq=0 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f10177fe640 1 -- 172.21.15.103:0/4251897901 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f5bb3da20 con 0x561f5bbe0ec0 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f10177fe640 10 client.25130 handle_command_reply: tid=0 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 client.25130 shutdown 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 -- 172.21.15.103:0/4251897901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 msgr2=0x561f5bbd6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbd6b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1028009230 tx=0x7f1028004270 comp rx=0 tx=0).stop 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 -- 172.21.15.103:0/4251897901 shutdown_connections 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x561f5bbe0ec0 0x561f5bbe3380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbde0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbd70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 --2- 172.21.15.103:0/4251897901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbd6b90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 -- 172.21.15.103:0/4251897901 >> 172.21.15.103:0/4251897901 conn(0x561f5bbc8010 msgr2=0x561f5bbc9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f10177fe640 0 client.25130 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 -- 172.21.15.103:0/4251897901 shutdown_connections 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.821+0000 7f1041098740 1 -- 172.21.15.103:0/4251897901 wait complete. 2024-09-05T02:04:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.822+0000 7f1041098740 20 client.25130 trim_cache size 0 max 16384 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1041098740 1 Processor -- start 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1041098740 1 -- start start 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1041098740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbc83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1041098740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1041098740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbc8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1041098740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10381a8a70 con 0x561f5bbd2370 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1041098740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f10381aa730 con 0x561f5bbcc9a0 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1041098740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10381a9140 con 0x561f5bbcd2e0 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1037fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbc83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f10377fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f103c8a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbc8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f10377fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbddfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:48722/0 (socket says 172.21.15.103:48722) 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f1037fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbc83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:51510/0 (socket says 172.21.15.103:51510) 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.843+0000 7f10377fe640 1 -- 172.21.15.103:0/2964918824 learned_addr learned my addr 172.21.15.103:0/2964918824 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f10377fe640 1 -- 172.21.15.103:0/2964918824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 msgr2=0x561f5bbc83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f10377fe640 1 --2- 172.21.15.103:0/2964918824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbc83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f10377fe640 1 -- 172.21.15.103:0/2964918824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 msgr2=0x561f5bbc8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f10377fe640 1 --2- 172.21.15.103:0/2964918824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbc8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f10377fe640 1 -- 172.21.15.103:0/2964918824 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5bbc7e30 con 0x561f5bbcd2e0 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f1037fff640 1 --2- 172.21.15.103:0/2964918824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbc83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f10377fe640 1 --2- 172.21.15.103:0/2964918824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbddfc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f102c0a4880 tx=0x7f102c097f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96686592e629fb51 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f101dffb640 1 -- 172.21.15.103:0/2964918824 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c099fa0 con 0x561f5bbcd2e0 2024-09-05T02:04:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f101dffb640 1 -- 172.21.15.103:0/2964918824 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f102c09b0f0 con 0x561f5bbcd2e0 2024-09-05T02:04:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.844+0000 7f101dffb640 1 -- 172.21.15.103:0/2964918824 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c097c50 con 0x561f5bbcd2e0 2024-09-05T02:04:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 -- 172.21.15.103:0/2964918824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 msgr2=0x561f5bbddfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 --2- 172.21.15.103:0/2964918824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbddfc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f102c0a4880 tx=0x7f102c097f40 comp rx=0 tx=0).stop 2024-09-05T02:04:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 -- 172.21.15.103:0/2964918824 shutdown_connections 2024-09-05T02:04:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 --2- 172.21.15.103:0/2964918824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbd2370 0x561f5bbc8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 --2- 172.21.15.103:0/2964918824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbcd2e0 0x561f5bbddfc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 --2- 172.21.15.103:0/2964918824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbc83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 -- 172.21.15.103:0/2964918824 >> 172.21.15.103:0/2964918824 conn(0x561f5bbc8010 msgr2=0x561f5bbcda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 -- 172.21.15.103:0/2964918824 shutdown_connections 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 -- 172.21.15.103:0/2964918824 wait complete. 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.845+0000 7f1041098740 1 -- start start 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1041098740 1 Processor -- start 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1041098740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbdb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1041098740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbcd2e0 0x561f5bbdbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1041098740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbd2370 0x561f5bbdcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1041098740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10381a9fa0 con 0x561f5bbcd2e0 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1041098740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f10381a9810 con 0x561f5bbcc9a0 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1041098740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10381a6a50 con 0x561f5bbd2370 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1041098740 10 client.0 _ll_register_callbacks cb 0x7f102c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1037fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbdb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1037fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbdb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:51526/0 (socket says 172.21.15.103:51526) 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1037fff640 1 -- 172.21.15.103:0/3010261 learned_addr learned my addr 172.21.15.103:0/3010261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f103c8a0640 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbd2370 0x561f5bbdcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f10377fe640 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbcd2e0 0x561f5bbdbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1037fff640 1 -- 172.21.15.103:0/3010261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbd2370 msgr2=0x561f5bbdcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1037fff640 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbd2370 0x561f5bbdcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1037fff640 1 -- 172.21.15.103:0/3010261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbcd2e0 msgr2=0x561f5bbdbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1037fff640 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbcd2e0 0x561f5bbdbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.846+0000 7f1037fff640 1 -- 172.21.15.103:0/3010261 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5bbdd1f0 con 0x561f5bbcc9a0 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.847+0000 7f1037fff640 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbdb690 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f102802aa00 tx=0x7f10280314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154e76f2cf8d12c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.847+0000 7f10177fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:04:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.847+0000 7f10177fe640 1 -- 172.21.15.103:0/3010261 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028059070 con 0x561f5bbcc9a0 2024-09-05T02:04:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.847+0000 7f1041098740 1 -- 172.21.15.103:0/3010261 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f102c09b0f0 con 0x561f5bbcc9a0 2024-09-05T02:04:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.847+0000 7f10177fe640 1 -- 172.21.15.103:0/3010261 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f10280035f0 con 0x561f5bbcc9a0 2024-09-05T02:04:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.847+0000 7f10177fe640 1 -- 172.21.15.103:0/3010261 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1028061050 con 0x561f5bbcc9a0 2024-09-05T02:04:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.847+0000 7f1041098740 10 client.25136 fetch_fsmap learned FSMap version 33 2024-09-05T02:04:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.847+0000 7f1041098740 1 -- 172.21.15.103:0/3010261 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561f5bc4d500 con 0x561f5bbcc9a0 2024-09-05T02:04:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.847+0000 7f10177fe640 1 -- 172.21.15.103:0/3010261 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f102c09b0f0 con 0x561f5bbcc9a0 2024-09-05T02:04:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f10177fe640 1 -- 172.21.15.103:0/3010261 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f102805c030 con 0x561f5bbcc9a0 2024-09-05T02:04:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f1041098740 10 client.25136 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:04:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f1041098740 10 client.25136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f1041098740 20 client.25136 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f1041098740 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x561f5bc4de20 0x561f5bc50240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f1041098740 4 client.25136 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:04:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f1041098740 1 -- 172.21.15.103:0/3010261 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f5bb37d70 con 0x561f5bc4de20 2024-09-05T02:04:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f10377fe640 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x561f5bc4de20 0x561f5bc50240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f10177fe640 10 client.25136 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.848+0000 7f10377fe640 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x561f5bc4de20 0x561f5bc50240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe2dc92e675c968 server_cookie=54f751d39e5e3f63 in_seq=0 out_seq=0 2024-09-05T02:04:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f10177fe640 1 -- 172.21.15.103:0/3010261 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f5bb37d70 con 0x561f5bc4de20 2024-09-05T02:04:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f10177fe640 10 client.25136 handle_command_reply: tid=0 2024-09-05T02:04:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 client.25136 shutdown 2024-09-05T02:04:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 -- 172.21.15.103:0/3010261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 msgr2=0x561f5bbdb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbdb690 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f102802aa00 tx=0x7f10280314a0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 -- 172.21.15.103:0/3010261 shutdown_connections 2024-09-05T02:04:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x561f5bc4de20 0x561f5bc50240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f5bbd2370 0x561f5bbdcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f5bbcd2e0 0x561f5bbdbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 --2- 172.21.15.103:0/3010261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561f5bbcc9a0 0x561f5bbdb690 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f10177fe640 0 client.25136 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 -- 172.21.15.103:0/3010261 >> 172.21.15.103:0/3010261 conn(0x561f5bbc8010 msgr2=0x561f5bbc69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 -- 172.21.15.103:0/3010261 shutdown_connections 2024-09-05T02:04:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 1 -- 172.21.15.103:0/3010261 wait complete. 2024-09-05T02:04:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.917+0000 7f1041098740 20 client.25136 trim_cache size 0 max 16384 2024-09-05T02:04:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:04:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:04:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:04:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:04:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 -- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f10200a2c30 msgr2=0x7f10200a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f10200a2c30 0x7f10200a50f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1030013fd0 tx=0x7f1030013560 comp rx=0 tx=0).stop 2024-09-05T02:04:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 -- 172.21.15.103:0/1884768870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10381025f0 msgr2=0x7f103819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10381025f0 0x7f103819cc70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e9f0 tx=0x7f102c00eeb0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 -- 172.21.15.103:0/1884768870 shutdown_connections 2024-09-05T02:04:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f103819d1b0 0x7f103819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10381025f0 0x7f103819cc70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f10200a2c30 0x7f10200a50f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 --2- 172.21.15.103:0/1884768870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1038101c30 0x7f103806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 -- 172.21.15.103:0/1884768870 >> 172.21.15.103:0/1884768870 conn(0x7f10380fc1e0 msgr2=0x7f1038105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 -- 172.21.15.103:0/1884768870 shutdown_connections 2024-09-05T02:04:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:39.918+0000 7f103e303640 1 -- 172.21.15.103:0/1884768870 wait complete. 2024-09-05T02:04:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:40 smithi167 ceph-mon[35122]: pgmap v417: 129 pgs: 129 active+clean; 70 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-09-05T02:04:40.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:40 smithi139 ceph-mon[34262]: pgmap v417: 129 pgs: 129 active+clean; 70 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-09-05T02:04:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:40 smithi103 ceph-mon[30902]: pgmap v417: 129 pgs: 129 active+clean; 70 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-09-05T02:04:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:42 smithi167 ceph-mon[35122]: pgmap v418: 129 pgs: 129 active+clean; 72 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-09-05T02:04:42.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:42 smithi139 ceph-mon[34262]: pgmap v418: 129 pgs: 129 active+clean; 72 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-09-05T02:04:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:42 smithi103 ceph-mon[30902]: pgmap v418: 129 pgs: 129 active+clean; 72 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-09-05T02:04:42.898 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.2 2024-09-05T02:04:42.899 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:04:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:04:43.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:43 smithi167 ceph-mon[35122]: pgmap v419: 129 pgs: 129 active+clean; 74 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 930 KiB/s wr, 149 op/s 2024-09-05T02:04:43.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:43 smithi139 ceph-mon[34262]: pgmap v419: 129 pgs: 129 active+clean; 74 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 930 KiB/s wr, 149 op/s 2024-09-05T02:04:43.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:43 smithi103 ceph-mon[30902]: pgmap v419: 129 pgs: 129 active+clean; 74 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 930 KiB/s wr, 149 op/s 2024-09-05T02:04:43.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.831+0000 7f7887772640 1 -- 172.21.15.103:0/469640751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880100a90 msgr2=0x7f7880100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:43.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.831+0000 7f7887772640 1 --2- 172.21.15.103:0/469640751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880100a90 0x7f7880100ef0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7870009930 tx=0x7f787002a420 comp rx=0 tx=0).stop 2024-09-05T02:04:43.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.831+0000 7f7887772640 1 -- 172.21.15.103:0/469640751 shutdown_connections 2024-09-05T02:04:43.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.831+0000 7f7887772640 1 --2- 172.21.15.103:0/469640751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7880101430 0x7f788010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:43.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.831+0000 7f7887772640 1 --2- 172.21.15.103:0/469640751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880100a90 0x7f7880100ef0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:43.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.831+0000 7f7887772640 1 --2- 172.21.15.103:0/469640751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7880106a90 0x7f7880106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:43.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.831+0000 7f7887772640 1 -- 172.21.15.103:0/469640751 >> 172.21.15.103:0/469640751 conn(0x7f78800fc7d0 msgr2=0x7f78800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:43.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.831+0000 7f7887772640 1 -- 172.21.15.103:0/469640751 shutdown_connections 2024-09-05T02:04:43.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.831+0000 7f7887772640 1 -- 172.21.15.103:0/469640751 wait complete. 2024-09-05T02:04:43.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7887772640 1 Processor -- start 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7887772640 1 -- start start 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7887772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7880100a90 0x7f7880073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7887772640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7880101430 0x7f788006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7887772640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880106a90 0x7f788006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7887772640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7880111e50 con 0x7f7880106a90 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7887772640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7880111cd0 con 0x7f7880100a90 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7887772640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7880111fd0 con 0x7f7880101430 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f788550e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7880100a90 0x7f7880073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f788550e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7880100a90 0x7f7880073aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:51542/0 (socket says 172.21.15.103:51542) 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f788550e640 1 -- 172.21.15.103:0/3144048629 learned_addr learned my addr 172.21.15.103:0/3144048629 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7885d0f640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880106a90 0x7f788006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:43.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.832+0000 7f7884d0d640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7880101430 0x7f788006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f788550e640 1 -- 172.21.15.103:0/3144048629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7880101430 msgr2=0x7f788006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f788550e640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7880101430 0x7f788006cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f788550e640 1 -- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880106a90 msgr2=0x7f788006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f788550e640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880106a90 0x7f788006d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f788550e640 1 -- 172.21.15.103:0/3144048629 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788006d730 con 0x7f7880100a90 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f7884d0d640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7880101430 0x7f788006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f788550e640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7880100a90 0x7f7880073aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9f0 tx=0x7f787c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f78767fc640 1 -- 172.21.15.103:0/3144048629 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c00cd80 con 0x7f7880100a90 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f78767fc640 1 -- 172.21.15.103:0/3144048629 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f787c0108e0 con 0x7f7880100a90 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.833+0000 7f78767fc640 1 -- 172.21.15.103:0/3144048629 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c018920 con 0x7f7880100a90 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.834+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78800715d0 con 0x7f7880100a90 2024-09-05T02:04:43.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.834+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78800719d0 con 0x7f7880100a90 2024-09-05T02:04:43.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.835+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78801021d0 con 0x7f7880100a90 2024-09-05T02:04:43.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.836+0000 7f78767fc640 1 -- 172.21.15.103:0/3144048629 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f787c010a90 con 0x7f7880100a90 2024-09-05T02:04:43.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.837+0000 7f78767fc640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f786c0a2c30 0x7f786c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:43.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.837+0000 7f7884d0d640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f786c0a2c30 0x7f786c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:43.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.837+0000 7f78767fc640 1 -- 172.21.15.103:0/3144048629 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f787c0cf660 con 0x7f7880100a90 2024-09-05T02:04:43.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.838+0000 7f7884d0d640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f786c0a2c30 0x7f786c0a50f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7870000c00 tx=0x7f7870009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:43.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:43.842+0000 7f78767fc640 1 -- 172.21.15.103:0/3144048629 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f787c09c210 con 0x7f7880100a90 2024-09-05T02:04:43.873 INFO:tasks.workunit.client.0.smithi103.stdout: 15 14379 37 8824 118 6421 116 2024-09-05T02:04:44.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.184+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f788006d900 con 0x7f7880100a90 2024-09-05T02:04:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.185+0000 7f78767fc640 1 -- 172.21.15.103:0/3144048629 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12012 (secure 0 0 0) 0x7f787c09bb10 con 0x7f7880100a90 2024-09-05T02:04:44.187 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:04:44.187 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":33,"btime":"2024-09-05T02:04:34:292690+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:04:33.568912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:04:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 33 2024-09-05T02:04:44.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.188+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f786c0a2c30 msgr2=0x7f786c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:44.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.188+0000 7f7887772640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f786c0a2c30 0x7f786c0a50f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7870000c00 tx=0x7f7870009610 comp rx=0 tx=0).stop 2024-09-05T02:04:44.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.188+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7880100a90 msgr2=0x7f7880073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:44.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.188+0000 7f7887772640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7880100a90 0x7f7880073aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9f0 tx=0x7f787c00eeb0 comp rx=0 tx=0).stop 2024-09-05T02:04:44.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.189+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 shutdown_connections 2024-09-05T02:04:44.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.189+0000 7f7887772640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880106a90 0x7f788006d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:44.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.189+0000 7f7887772640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f786c0a2c30 0x7f786c0a50f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:44.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.189+0000 7f7887772640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7880101430 0x7f788006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:44.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.189+0000 7f7887772640 1 --2- 172.21.15.103:0/3144048629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7880100a90 0x7f7880073aa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:44.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.189+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 >> 172.21.15.103:0/3144048629 conn(0x7f78800fc7d0 msgr2=0x7f78800fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:44.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.189+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 shutdown_connections 2024-09-05T02:04:44.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:44.189+0000 7f7887772640 1 -- 172.21.15.103:0/3144048629 wait complete. 2024-09-05T02:04:44.578 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:44 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3144048629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:04:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:44 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3144048629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:04:44.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:44 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3144048629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:04:45.053 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 144.2 --await-for 45.0' 2024-09-05T02:04:45.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 144.2 --await-for 45.0 2024-09-05T02:04:45.399 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:04:45.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:45 smithi167 ceph-mon[35122]: pgmap v420: 129 pgs: 129 active+clean; 76 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-09-05T02:04:45.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:45 smithi139 ceph-mon[34262]: pgmap v420: 129 pgs: 129 active+clean; 76 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-09-05T02:04:45.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:45 smithi103 ceph-mon[30902]: pgmap v420: 129 pgs: 129 active+clean; 76 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-09-05T02:04:45.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.992+0000 7fd2423ee640 1 -- 172.21.15.103:0/1784169067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c106a80 msgr2=0x7fd23c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:45.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.992+0000 7fd2423ee640 1 --2- 172.21.15.103:0/1784169067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c106a80 0x7fd23c106e60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009890 tx=0x7fd22c02a380 comp rx=0 tx=0).stop 2024-09-05T02:04:45.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.992+0000 7fd2423ee640 1 -- 172.21.15.103:0/1784169067 shutdown_connections 2024-09-05T02:04:45.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.992+0000 7fd2423ee640 1 --2- 172.21.15.103:0/1784169067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd23c101420 0x7fd23c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:45.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.992+0000 7fd2423ee640 1 --2- 172.21.15.103:0/1784169067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd23c100a80 0x7fd23c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:45.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.992+0000 7fd2423ee640 1 --2- 172.21.15.103:0/1784169067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c106a80 0x7fd23c106e60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:45.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.992+0000 7fd2423ee640 1 -- 172.21.15.103:0/1784169067 >> 172.21.15.103:0/1784169067 conn(0x7fd23c0fc7a0 msgr2=0x7fd23c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:45.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.993+0000 7fd2423ee640 1 -- 172.21.15.103:0/1784169067 shutdown_connections 2024-09-05T02:04:45.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.993+0000 7fd2423ee640 1 -- 172.21.15.103:0/1784169067 wait complete. 2024-09-05T02:04:45.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.993+0000 7fd2423ee640 1 Processor -- start 2024-09-05T02:04:45.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.993+0000 7fd2423ee640 1 -- start start 2024-09-05T02:04:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd2423ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c100a80 0x7fd23c110960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd2423ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd23c101420 0x7fd23c110ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd2423ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd23c106a80 0x7fd23c1113e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd2423ee640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c06ab20 con 0x7fd23c106a80 2024-09-05T02:04:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd2423ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c06a9a0 con 0x7fd23c101420 2024-09-05T02:04:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd2423ee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c06aca0 con 0x7fd23c100a80 2024-09-05T02:04:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd23b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd23c101420 0x7fd23c110ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c100a80 0x7fd23c110960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd23b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd23c101420 0x7fd23c110ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:51556/0 (socket says 172.21.15.103:51556) 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd23b7fe640 1 -- 172.21.15.103:0/305956971 learned_addr learned my addr 172.21.15.103:0/305956971 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c100a80 0x7fd23c110960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:48764/0 (socket says 172.21.15.103:48764) 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.994+0000 7fd24098b640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd23c106a80 0x7fd23c1113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd23b7fe640 1 -- 172.21.15.103:0/305956971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c100a80 msgr2=0x7fd23c110960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c100a80 0x7fd23c110960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd23b7fe640 1 -- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd23c106a80 msgr2=0x7fd23c1113e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd23c106a80 0x7fd23c1113e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd23b7fe640 1 -- 172.21.15.103:0/305956971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23c072880 con 0x7fd23c101420 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd24098b640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd23c106a80 0x7fd23c1113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd23c101420 0x7fd23c110ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd23000e9c0 tx=0x7fd23000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd23bfff640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c100a80 0x7fd23c110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd2397fa640 1 -- 172.21.15.103:0/305956971 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23000ccc0 con 0x7fd23c101420 2024-09-05T02:04:45.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd2397fa640 1 -- 172.21.15.103:0/305956971 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2300054d0 con 0x7fd23c101420 2024-09-05T02:04:45.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.995+0000 7fd2423ee640 1 -- 172.21.15.103:0/305956971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23c072b70 con 0x7fd23c101420 2024-09-05T02:04:45.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.996+0000 7fd2423ee640 1 -- 172.21.15.103:0/305956971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23c073050 con 0x7fd23c101420 2024-09-05T02:04:45.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.996+0000 7fd2397fa640 1 -- 172.21.15.103:0/305956971 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230010430 con 0x7fd23c101420 2024-09-05T02:04:45.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.998+0000 7fd2397fa640 1 -- 172.21.15.103:0/305956971 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd2300105d0 con 0x7fd23c101420 2024-09-05T02:04:46.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2447c7740 1 Processor -- start 2024-09-05T02:04:46.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2447c7740 1 -- start start 2024-09-05T02:04:46.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2397fa640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2240a2cd0 0x7fd2240a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4878b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4883400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d4885d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2447c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5612d47de2a0 con 0x5612d4878760 2024-09-05T02:04:46.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2447c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5612d4785b10 con 0x5612d4879080 2024-09-05T02:04:46.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2447c7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5612d47da260 con 0x5612d4883940 2024-09-05T02:04:46.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd2397fa640 1 -- 172.21.15.103:0/305956971 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd2300cf600 con 0x7fd23c101420 2024-09-05T02:04:46.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd23b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4883400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd23b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4883400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:51564/0 (socket says 172.21.15.103:51564) 2024-09-05T02:04:46.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd23b7fe640 1 -- 172.21.15.103:0/4081451049 learned_addr learned my addr 172.21.15.103:0/4081451049 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:46.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd23bfff640 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4878b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd24098b640 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d4885d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd23b7fe640 1 -- 172.21.15.103:0/4081451049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 msgr2=0x5612d4885d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d4885d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:45.999+0000 7fd23b7fe640 1 -- 172.21.15.103:0/4081451049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 msgr2=0x5612d4878b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.000+0000 7fd24098b640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2240a2cd0 0x7fd2240a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.000+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4878b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.000+0000 7fd23b7fe640 1 -- 172.21.15.103:0/4081451049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d454fb90 con 0x5612d4879080 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.000+0000 7fd23bfff640 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4878b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.000+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4883400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd23000c6a0 tx=0x7fd230009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540858a18fc59ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.000+0000 7fd24098b640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2240a2cd0 0x7fd2240a5190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5612d4876070 tx=0x7fd234004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.000+0000 7fd2227fc640 1 -- 172.21.15.103:0/4081451049 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230087340 con 0x5612d4879080 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.000+0000 7fd2227fc640 1 -- 172.21.15.103:0/4081451049 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd230099120 con 0x5612d4879080 2024-09-05T02:04:46.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.000+0000 7fd2227fc640 1 -- 172.21.15.103:0/4081451049 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23009aab0 con 0x5612d4879080 2024-09-05T02:04:46.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.001+0000 7fd2447c7740 1 -- 172.21.15.103:0/4081451049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 msgr2=0x5612d4883400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:46.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.001+0000 7fd2447c7740 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4883400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd23000c6a0 tx=0x7fd230009f90 comp rx=0 tx=0).stop 2024-09-05T02:04:46.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.001+0000 7fd2447c7740 1 -- 172.21.15.103:0/4081451049 shutdown_connections 2024-09-05T02:04:46.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.001+0000 7fd2447c7740 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d4885d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.001+0000 7fd2447c7740 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4883400 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.001+0000 7fd2447c7740 1 --2- 172.21.15.103:0/4081451049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4878b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.001+0000 7fd2447c7740 1 -- 172.21.15.103:0/4081451049 >> 172.21.15.103:0/4081451049 conn(0x5612d4874cf0 msgr2=0x5612d48750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.001+0000 7fd2447c7740 1 -- 172.21.15.103:0/4081451049 shutdown_connections 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.002+0000 7fd2447c7740 1 -- 172.21.15.103:0/4081451049 wait complete. 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.002+0000 7fd2447c7740 1 -- start start 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.002+0000 7fd2447c7740 1 Processor -- start 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4878760 0x5612d4880850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4879080 0x5612d4880d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d48812d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd2447c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5612d4886510 con 0x5612d4879080 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd2447c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5612d47666e0 con 0x5612d4878760 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd2447c7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5612d4886690 con 0x5612d4883940 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd2447c7740 10 client.0 _ll_register_callbacks cb 0x5612d478aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4879080 0x5612d4880d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4879080 0x5612d4880d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48710/0 (socket says 172.21.15.103:48710) 2024-09-05T02:04:46.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23b7fe640 1 -- 172.21.15.103:0/3891512428 learned_addr learned my addr 172.21.15.103:0/3891512428 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:46.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd24098b640 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d48812d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23bfff640 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4878760 0x5612d4880850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23b7fe640 1 -- 172.21.15.103:0/3891512428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 msgr2=0x5612d48812d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:46.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d48812d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23b7fe640 1 -- 172.21.15.103:0/3891512428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4878760 msgr2=0x5612d4880850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:46.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4878760 0x5612d4880850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23b7fe640 1 -- 172.21.15.103:0/3891512428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d488fa90 con 0x5612d4879080 2024-09-05T02:04:46.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd24098b640 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d48812d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4879080 0x5612d4880d90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd23009c4c0 tx=0x7fd230005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb32ac030ad4866 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.003+0000 7fd21b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.004+0000 7fd21b7fe640 1 -- 172.21.15.103:0/3891512428 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230096220 con 0x5612d4879080 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.004+0000 7fd21b7fe640 1 -- 172.21.15.103:0/3891512428 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd2300963c0 con 0x5612d4879080 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.004+0000 7fd2447c7740 1 -- 172.21.15.103:0/3891512428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd230099120 con 0x5612d4879080 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.004+0000 7fd21b7fe640 1 -- 172.21.15.103:0/3891512428 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd23009aab0 con 0x5612d4879080 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.004+0000 7fd2447c7740 10 client.15687 fetch_fsmap learned FSMap version 33 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.004+0000 7fd2447c7740 1 -- 172.21.15.103:0/3891512428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5612d4892420 con 0x5612d4879080 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.004+0000 7fd21b7fe640 1 -- 172.21.15.103:0/3891512428 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd230096070 con 0x5612d4879080 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.004+0000 7fd21b7fe640 1 -- 172.21.15.103:0/3891512428 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fd230004040 con 0x5612d4879080 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.005+0000 7fd2447c7740 10 client.15687 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:04:46.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.005+0000 7fd2447c7740 10 client.15687 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:04:46.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.005+0000 7fd2447c7740 20 client.15687 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:46.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.005+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5612d4893770 0x5612d4895b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.005+0000 7fd2447c7740 4 client.15687 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:04:46.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.005+0000 7fd2447c7740 1 -- 172.21.15.103:0/3891512428 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd230099120 con 0x5612d4893770 2024-09-05T02:04:46.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.005+0000 7fd23bfff640 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5612d4893770 0x5612d4895b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.005+0000 7fd23bfff640 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5612d4893770 0x5612d4895b60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12fb9a4cbcb05662 server_cookie=4071b9ab52e37521 in_seq=0 out_seq=0 2024-09-05T02:04:46.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.005+0000 7fd21b7fe640 10 client.15687 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:46.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd21b7fe640 1 -- 172.21.15.103:0/3891512428 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd230099120 con 0x5612d4893770 2024-09-05T02:04:46.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd21b7fe640 10 client.15687 handle_command_reply: tid=0 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd2447c7740 1 client.15687 shutdown 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd2447c7740 1 -- 172.21.15.103:0/3891512428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4879080 msgr2=0x5612d4880d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4879080 0x5612d4880d90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd23009c4c0 tx=0x7fd230005760 comp rx=0 tx=0).stop 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd2447c7740 1 -- 172.21.15.103:0/3891512428 shutdown_connections 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5612d4893770 0x5612d4895b60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d48812d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4879080 0x5612d4880d90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3891512428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4878760 0x5612d4880850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd21b7fe640 0 client.15687 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.006+0000 7fd2447c7740 1 -- 172.21.15.103:0/3891512428 >> 172.21.15.103:0/3891512428 conn(0x5612d4874cf0 msgr2=0x5612d487aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.007+0000 7fd2447c7740 1 -- 172.21.15.103:0/3891512428 shutdown_connections 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.007+0000 7fd2447c7740 1 -- 172.21.15.103:0/3891512428 wait complete. 2024-09-05T02:04:46.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.007+0000 7fd2447c7740 20 client.15687 trim_cache size 0 max 16384 2024-09-05T02:04:46.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.030+0000 7fd2447c7740 1 Processor -- start 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.030+0000 7fd2447c7740 1 -- start start 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4878760 0x5612d4881120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4873d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4883940 0x5612d487dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd2447c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd22800fc20 con 0x5612d4883940 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd2447c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd2280118e0 con 0x5612d4879080 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd2447c7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2280102f0 con 0x5612d4878760 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4878760 0x5612d4881120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4878760 0x5612d4881120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:48796/0 (socket says 172.21.15.103:48796) 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23bfff640 1 -- 172.21.15.103:0/3689591831 learned_addr learned my addr 172.21.15.103:0/3689591831 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd24098b640 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4883940 0x5612d487dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4873d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23bfff640 1 -- 172.21.15.103:0/3689591831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 msgr2=0x5612d4873d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23bfff640 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4873d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23bfff640 1 -- 172.21.15.103:0/3689591831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4883940 msgr2=0x5612d487dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23bfff640 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4883940 0x5612d487dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23bfff640 1 -- 172.21.15.103:0/3689591831 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d487f800 con 0x5612d4878760 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd24098b640 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4883940 0x5612d487dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.031+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4873d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.032+0000 7fd23bfff640 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4878760 0x5612d4881120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd22c000c00 tx=0x7fd22c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb4655a8402d1b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.032+0000 7fd221ffb640 1 -- 172.21.15.103:0/3689591831 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c037040 con 0x5612d4878760 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.032+0000 7fd221ffb640 1 -- 172.21.15.103:0/3689591831 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd22c030450 con 0x5612d4878760 2024-09-05T02:04:46.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.032+0000 7fd221ffb640 1 -- 172.21.15.103:0/3689591831 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c003950 con 0x5612d4878760 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 -- 172.21.15.103:0/3689591831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4878760 msgr2=0x5612d4881120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4878760 0x5612d4881120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd22c000c00 tx=0x7fd22c005f90 comp rx=0 tx=0).stop 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 -- 172.21.15.103:0/3689591831 shutdown_connections 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4883940 0x5612d487dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4873d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 --2- 172.21.15.103:0/3689591831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4878760 0x5612d4881120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 -- 172.21.15.103:0/3689591831 >> 172.21.15.103:0/3689591831 conn(0x5612d4874cf0 msgr2=0x5612d4873650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 -- 172.21.15.103:0/3689591831 shutdown_connections 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 -- 172.21.15.103:0/3689591831 wait complete. 2024-09-05T02:04:46.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.033+0000 7fd2447c7740 1 -- start start 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd2447c7740 1 Processor -- start 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4896b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4897090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4896b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd2447c7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d48975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4896b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48724/0 (socket says 172.21.15.103:48724) 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd23bfff640 1 -- 172.21.15.103:0/351443691 learned_addr learned my addr 172.21.15.103:0/351443691 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd228011150 con 0x5612d4878760 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd2280109c0 con 0x5612d4879080 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd22800dc00 con 0x5612d4883940 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4897090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd24098b640 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d48975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd2447c7740 10 client.0 _ll_register_callbacks cb 0x7fd2300a4860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd23bfff640 1 -- 172.21.15.103:0/351443691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 msgr2=0x5612d48975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd23bfff640 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d48975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd23bfff640 1 -- 172.21.15.103:0/351443691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 msgr2=0x5612d4897090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd23bfff640 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4897090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.034+0000 7fd23bfff640 1 -- 172.21.15.103:0/351443691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d48fd300 con 0x5612d4878760 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.035+0000 7fd23bfff640 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4896b50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd22c030dc0 tx=0x7fd22c030df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7886a2dbbca49d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.035+0000 7fd21b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.035+0000 7fd21b7fe640 1 -- 172.21.15.103:0/351443691 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c04d020 con 0x5612d4878760 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.035+0000 7fd21b7fe640 1 -- 172.21.15.103:0/351443691 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd22c004240 con 0x5612d4878760 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.035+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd22c030450 con 0x5612d4878760 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.035+0000 7fd21b7fe640 1 -- 172.21.15.103:0/351443691 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd22c058050 con 0x5612d4878760 2024-09-05T02:04:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.035+0000 7fd2447c7740 10 client.15693 fetch_fsmap learned FSMap version 33 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.035+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5612d48fe5a0 con 0x5612d4878760 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.036+0000 7fd21b7fe640 1 -- 172.21.15.103:0/351443691 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd22c030450 con 0x5612d4878760 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.036+0000 7fd21b7fe640 1 -- 172.21.15.103:0/351443691 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fd22c049470 con 0x5612d4878760 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.036+0000 7fd2447c7740 10 client.15693 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.036+0000 7fd2447c7740 10 client.15693 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.036+0000 7fd2447c7740 20 client.15693 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.036+0000 7fd2447c7740 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5612d48fed90 0x5612d49011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.036+0000 7fd2447c7740 4 client.15693 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.2, "await_for": 45.0}] 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.036+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.2, "await_for": 45.0}) -- 0x5612d47eb9d0 con 0x5612d48fed90 2024-09-05T02:04:46.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.036+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5612d48fed90 0x5612d49011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:46.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.037+0000 7fd21b7fe640 10 client.15693 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:46.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:46.037+0000 7fd23b7fe640 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5612d48fed90 0x5612d49011d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88a026b4f21c2702 server_cookie=265bc1070a4d0bab in_seq=0 out_seq=0 2024-09-05T02:04:48.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd21b7fe640 1 -- 172.21.15.103:0/351443691 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5612d47eb9d0 con 0x5612d48fed90 2024-09-05T02:04:48.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd21b7fe640 10 client.15693 handle_command_reply: tid=0 2024-09-05T02:04:48.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd2447c7740 1 client.15693 shutdown 2024-09-05T02:04:48.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 msgr2=0x5612d4896b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:48.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd2447c7740 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4896b50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd22c030dc0 tx=0x7fd22c030df0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 shutdown_connections 2024-09-05T02:04:48.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd2447c7740 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5612d48fed90 0x5612d49011d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd2447c7740 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612d4883940 0x5612d48975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd2447c7740 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612d4879080 0x5612d4897090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.261+0000 7fd2447c7740 1 --2- 172.21.15.103:0/351443691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612d4878760 0x5612d4896b50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.260+0000 7fd21b7fe640 0 client.15693 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:48.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.261+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 >> 172.21.15.103:0/351443691 conn(0x5612d4874cf0 msgr2=0x5612d488f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:48.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.261+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 shutdown_connections 2024-09-05T02:04:48.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.261+0000 7fd2447c7740 1 -- 172.21.15.103:0/351443691 wait complete. 2024-09-05T02:04:48.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.261+0000 7fd2447c7740 20 client.15693 trim_cache size 0 max 16384 2024-09-05T02:04:48.265 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:04:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 33, 2024-09-05T02:04:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:04:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 6, 2024-09-05T02:04:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:04:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "bd6d4922": { 2024-09-05T02:04:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 6, 2024-09-05T02:04:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:04:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:04:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:04:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:04:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:04:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:04:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 144.2, 2024-09-05T02:04:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:04:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:04:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:04:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:04:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:04:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:04:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:04:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:04:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:04:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:04:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:04:48.268 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:04:48.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.261+0000 7fd2423ee640 1 -- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2240a2cd0 msgr2=0x7fd2240a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:48.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.261+0000 7fd2423ee640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2240a2cd0 0x7fd2240a5190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5612d4876070 tx=0x7fd234004ab0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.261+0000 7fd2423ee640 1 -- 172.21.15.103:0/305956971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd23c101420 msgr2=0x7fd23c110ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.261+0000 7fd2423ee640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd23c101420 0x7fd23c110ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd23000e9c0 tx=0x7fd23000ee80 comp rx=0 tx=0).stop 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.262+0000 7fd2423ee640 1 -- 172.21.15.103:0/305956971 shutdown_connections 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.262+0000 7fd2423ee640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd23c106a80 0x7fd23c1113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.262+0000 7fd2423ee640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd23c101420 0x7fd23c110ea0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.262+0000 7fd2423ee640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd2240a2cd0 0x7fd2240a5190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.262+0000 7fd2423ee640 1 --2- 172.21.15.103:0/305956971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd23c100a80 0x7fd23c110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.262+0000 7fd2423ee640 1 -- 172.21.15.103:0/305956971 >> 172.21.15.103:0/305956971 conn(0x7fd23c0fc7a0 msgr2=0x7fd23c0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.262+0000 7fd2423ee640 1 -- 172.21.15.103:0/305956971 shutdown_connections 2024-09-05T02:04:48.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:04:48.262+0000 7fd2423ee640 1 -- 172.21.15.103:0/305956971 wait complete. 2024-09-05T02:04:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:48 smithi167 ceph-mon[35122]: pgmap v421: 129 pgs: 129 active+clean; 78 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-09-05T02:04:48.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:48 smithi103 ceph-mon[30902]: pgmap v421: 129 pgs: 129 active+clean; 78 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-09-05T02:04:48.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:48 smithi139 ceph-mon[34262]: pgmap v421: 129 pgs: 129 active+clean; 78 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-09-05T02:04:49.120 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bd6d4922', quiesce took 6.2 seconds. Will release after: 18.0 2024-09-05T02:04:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:50 smithi167 ceph-mon[35122]: pgmap v422: 129 pgs: 129 active+clean; 83 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-05T02:04:50.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:50 smithi139 ceph-mon[34262]: pgmap v422: 129 pgs: 129 active+clean; 83 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-05T02:04:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:50 smithi103 ceph-mon[30902]: pgmap v422: 129 pgs: 129 active+clean; 83 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-05T02:04:52.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:52 smithi167 ceph-mon[35122]: pgmap v423: 129 pgs: 129 active+clean; 89 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.8 MiB/s wr, 381 op/s 2024-09-05T02:04:52.964 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:52 smithi103 ceph-mon[30902]: pgmap v423: 129 pgs: 129 active+clean; 89 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.8 MiB/s wr, 381 op/s 2024-09-05T02:04:52.965 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:52 smithi139 ceph-mon[34262]: pgmap v423: 129 pgs: 129 active+clean; 89 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.8 MiB/s wr, 381 op/s 2024-09-05T02:04:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 -- 172.21.15.103:0/4143097679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 msgr2=0x7ffba8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 --2- 172.21.15.103:0/4143097679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 0x7ffba8101f60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009900 tx=0x7ffb9802a410 comp rx=0 tx=0).stop 2024-09-05T02:04:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 -- 172.21.15.103:0/4143097679 shutdown_connections 2024-09-05T02:04:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 --2- 172.21.15.103:0/4143097679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba8103710 0x7ffba8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 --2- 172.21.15.103:0/4143097679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffba8102d50 0x7ffba81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 --2- 172.21.15.103:0/4143097679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 0x7ffba8101f60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 -- 172.21.15.103:0/4143097679 >> 172.21.15.103:0/4143097679 conn(0x7ffba80fd2e0 msgr2=0x7ffba80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 -- 172.21.15.103:0/4143097679 shutdown_connections 2024-09-05T02:04:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 -- 172.21.15.103:0/4143097679 wait complete. 2024-09-05T02:04:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 Processor -- start 2024-09-05T02:04:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.019+0000 7ffbae22e640 1 -- start start 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffbae22e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 0x7ffba8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffbae22e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffba8102d50 0x7ffba80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffbae22e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba8103710 0x7ffba8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffbae22e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffba810d670 con 0x7ffba8101b60 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffbae22e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffba810d4f0 con 0x7ffba8102d50 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffbae22e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffba810d7f0 con 0x7ffba8103710 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 0x7ffba8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 0x7ffba8075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49430/0 (socket says 172.21.15.103:49430) 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba77fe640 1 -- 172.21.15.103:0/1200498088 learned_addr learned my addr 172.21.15.103:0/1200498088 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba7fff640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba8103710 0x7ffba8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba6ffd640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffba8102d50 0x7ffba80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba77fe640 1 -- 172.21.15.103:0/1200498088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba8103710 msgr2=0x7ffba8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba77fe640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba8103710 0x7ffba8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba77fe640 1 -- 172.21.15.103:0/1200498088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffba8102d50 msgr2=0x7ffba80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba77fe640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffba8102d50 0x7ffba80745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba77fe640 1 -- 172.21.15.103:0/1200498088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba80753b0 con 0x7ffba8101b60 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba7fff640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba8103710 0x7ffba8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.020+0000 7ffba77fe640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 0x7ffba8075fa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009230 tx=0x7ffb98004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.021+0000 7ffba4ff9640 1 -- 172.21.15.103:0/1200498088 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9803f030 con 0x7ffba8101b60 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.021+0000 7ffbae22e640 1 -- 172.21.15.103:0/1200498088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba81a1cd0 con 0x7ffba8101b60 2024-09-05T02:04:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.021+0000 7ffba4ff9640 1 -- 172.21.15.103:0/1200498088 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb98003cb0 con 0x7ffba8101b60 2024-09-05T02:04:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.021+0000 7ffbae22e640 1 -- 172.21.15.103:0/1200498088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba81a21c0 con 0x7ffba8101b60 2024-09-05T02:04:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.021+0000 7ffba4ff9640 1 -- 172.21.15.103:0/1200498088 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98049060 con 0x7ffba8101b60 2024-09-05T02:04:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffba4ff9640 1 -- 172.21.15.103:0/1200498088 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffb98042030 con 0x7ffba8101b60 2024-09-05T02:04:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffbb0fcb740 1 Processor -- start 2024-09-05T02:04:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffbb0fcb740 1 -- start start 2024-09-05T02:04:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b39a0 0x55d70d3b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b42e0 0x55d70d3b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3bb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d70d3172f0 con 0x55d70d3b9370 2024-09-05T02:04:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d70d2bf270 con 0x55d70d3b42e0 2024-09-05T02:04:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d70d3132b0 con 0x55d70d3b39a0 2024-09-05T02:04:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.023+0000 7ffba4ff9640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffb900a2c30 0x7ffb900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3bb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3bb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49440/0 (socket says 172.21.15.103:49440) 2024-09-05T02:04:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba7fff640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffb900a2c30 0x7ffb900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba6ffd640 1 -- 172.21.15.103:0/1728150094 learned_addr learned my addr 172.21.15.103:0/1728150094 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba7fff640 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b39a0 0x55d70d3b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba77fe640 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b42e0 0x55d70d3b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba4ff9640 1 -- 172.21.15.103:0/1200498088 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffb980efec0 con 0x7ffba8101b60 2024-09-05T02:04:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba6ffd640 1 -- 172.21.15.103:0/1728150094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b39a0 msgr2=0x55d70d3b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba6ffd640 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b39a0 0x55d70d3b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba6ffd640 1 -- 172.21.15.103:0/1728150094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b42e0 msgr2=0x55d70d3b8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba6ffd640 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b42e0 0x55d70d3b8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba6ffd640 1 -- 172.21.15.103:0/1728150094 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d70d2be340 con 0x55d70d3b9370 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba77fe640 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b42e0 0x55d70d3b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba7fff640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffb900a2c30 0x7ffb900a50f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffba0005fd0 tx=0x7ffba000ad30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffba6ffd640 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3bb830 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00bb10 tx=0x7ffb9c00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648a272a41fb7f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffb8dffb640 1 -- 172.21.15.103:0/1728150094 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c017420 con 0x55d70d3b9370 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffb8dffb640 1 -- 172.21.15.103:0/1728150094 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb9c017a70 con 0x55d70d3b9370 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.024+0000 7ffb8dffb640 1 -- 172.21.15.103:0/1728150094 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00f980 con 0x55d70d3b9370 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/1728150094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 msgr2=0x55d70d3bb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3bb830 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00bb10 tx=0x7ffb9c00d340 comp rx=0 tx=0).stop 2024-09-05T02:04:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/1728150094 shutdown_connections 2024-09-05T02:04:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3bb830 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b42e0 0x55d70d3b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/1728150094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b39a0 0x55d70d3b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/1728150094 >> 172.21.15.103:0/1728150094 conn(0x55d70d3af010 msgr2=0x55d70d3af410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/1728150094 shutdown_connections 2024-09-05T02:04:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/1728150094 wait complete. 2024-09-05T02:04:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.025+0000 7ffbb0fcb740 1 -- start start 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffbb0fcb740 1 Processor -- start 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b42e0 0x55d70d3be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d70d3bc340 con 0x55d70d3b9370 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d70d2a0290 con 0x55d70d3b39a0 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d70d3bc4c0 con 0x55d70d3b42e0 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffbb0fcb740 10 client.0 _ll_register_callbacks cb 0x55d70d2d1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49452/0 (socket says 172.21.15.103:49452) 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba7fff640 1 -- 172.21.15.103:0/143959392 learned_addr learned my addr 172.21.15.103:0/143959392 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba6ffd640 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b42e0 0x55d70d3be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba7fff640 1 -- 172.21.15.103:0/143959392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b42e0 msgr2=0x55d70d3be0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba7fff640 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b42e0 0x55d70d3be0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba7fff640 1 -- 172.21.15.103:0/143959392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 msgr2=0x55d70d3bdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba7fff640 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3bdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.026+0000 7ffba7fff640 1 -- 172.21.15.103:0/143959392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70d02a210 con 0x55d70d3b9370 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffba7fff640 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3c50e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ffba000ff50 tx=0x7ffba0005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5eb9268a23efe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffb86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffb86ffd640 1 -- 172.21.15.103:0/143959392 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba002a680 con 0x55d70d3b9370 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffb86ffd640 1 -- 172.21.15.103:0/143959392 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffba002a820 con 0x55d70d3b9370 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/143959392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb9c017a70 con 0x55d70d3b9370 2024-09-05T02:04:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffb86ffd640 1 -- 172.21.15.103:0/143959392 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffba002aa60 con 0x55d70d3b9370 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffbb0fcb740 10 client.15705 fetch_fsmap learned FSMap version 33 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/143959392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d70d0f6620 con 0x55d70d3b9370 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffb86ffd640 1 -- 172.21.15.103:0/143959392 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb9c017a70 con 0x55d70d3b9370 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffb86ffd640 1 -- 172.21.15.103:0/143959392 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7ffba001fbc0 con 0x55d70d3b9370 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffbb0fcb740 10 client.15705 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.027+0000 7ffbb0fcb740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.028+0000 7ffbb0fcb740 20 client.15705 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.028+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d70d3c97e0 0x55d70d3cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.028+0000 7ffbb0fcb740 4 client.15705 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.028+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/143959392 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d70d324a20 con 0x55d70d3c97e0 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.028+0000 7ffba77fe640 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d70d3c97e0 0x55d70d3cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.028+0000 7ffba77fe640 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d70d3c97e0 0x55d70d3cbc60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5e1759b72943a0 server_cookie=32bdb885c045237 in_seq=0 out_seq=0 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.028+0000 7ffb86ffd640 10 client.15705 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffb86ffd640 1 -- 172.21.15.103:0/143959392 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d70d324a20 con 0x55d70d3c97e0 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffb86ffd640 10 client.15705 handle_command_reply: tid=0 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 client.15705 shutdown 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/143959392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 msgr2=0x55d70d3c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3c50e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ffba000ff50 tx=0x7ffba0005f40 comp rx=0 tx=0).stop 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/143959392 shutdown_connections 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d70d3c97e0 0x55d70d3cbc60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3c50e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b42e0 0x55d70d3be0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffb86ffd640 0 client.15705 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/143959392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3bdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/143959392 >> 172.21.15.103:0/143959392 conn(0x55d70d3af010 msgr2=0x55d70d3b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/143959392 shutdown_connections 2024-09-05T02:04:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/143959392 wait complete. 2024-09-05T02:04:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.029+0000 7ffbb0fcb740 20 client.15705 trim_cache size 0 max 16384 2024-09-05T02:04:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffbb0fcb740 1 Processor -- start 2024-09-05T02:04:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffbb0fcb740 1 -- start start 2024-09-05T02:04:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b42e0 0x55d70d3c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffba81ac7c0 con 0x55d70d3b9370 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffba81ae480 con 0x55d70d3b39a0 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffba81ace90 con 0x55d70d3b42e0 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3d0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49466/0 (socket says 172.21.15.103:49466) 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba7fff640 1 -- 172.21.15.103:0/4288813643 learned_addr learned my addr 172.21.15.103:0/4288813643 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba7fff640 1 -- 172.21.15.103:0/4288813643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b42e0 msgr2=0x55d70d3c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba7fff640 1 --2- 172.21.15.103:0/4288813643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b42e0 0x55d70d3c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba7fff640 1 -- 172.21.15.103:0/4288813643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 msgr2=0x55d70d3ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba77fe640 1 --2- 172.21.15.103:0/4288813643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba7fff640 1 --2- 172.21.15.103:0/4288813643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba7fff640 1 -- 172.21.15.103:0/4288813643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d70d3b47c0 con 0x55d70d3b9370 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.044+0000 7ffba7fff640 1 --2- 172.21.15.103:0/4288813643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3d0520 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ffba000abf0 tx=0x7ffba000a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87d9af29a9cb29f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffb8d7fa640 1 -- 172.21.15.103:0/4288813643 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0012030 con 0x55d70d3b9370 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffb8d7fa640 1 -- 172.21.15.103:0/4288813643 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba003c070 con 0x55d70d3b9370 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffb8d7fa640 1 -- 172.21.15.103:0/4288813643 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba00289d0 con 0x55d70d3b9370 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/4288813643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 msgr2=0x55d70d3d0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/4288813643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3d0520 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ffba000abf0 tx=0x7ffba000a880 comp rx=0 tx=0).stop 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/4288813643 shutdown_connections 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/4288813643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b9370 0x55d70d3d0520 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/4288813643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b42e0 0x55d70d3c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/4288813643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/4288813643 >> 172.21.15.103:0/4288813643 conn(0x55d70d3af010 msgr2=0x55d70d3ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/4288813643 shutdown_connections 2024-09-05T02:04:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/4288813643 wait complete. 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.045+0000 7ffbb0fcb740 1 -- start start 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffbb0fcb740 1 Processor -- start 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b42e0 0x55d70d3c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffbb0fcb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b9370 0x55d70d3c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffba81adcf0 con 0x55d70d3b42e0 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffba81ad560 con 0x55d70d3b39a0 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffbb0fcb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffba81aa7a0 con 0x55d70d3b9370 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffbb0fcb740 10 client.0 _ll_register_callbacks cb 0x7ffba000fc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b42e0 0x55d70d3c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b42e0 0x55d70d3c2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49472/0 (socket says 172.21.15.103:49472) 2024-09-05T02:04:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba6ffd640 1 -- 172.21.15.103:0/2666768112 learned_addr learned my addr 172.21.15.103:0/2666768112 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba7fff640 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b9370 0x55d70d3c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba77fe640 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba6ffd640 1 -- 172.21.15.103:0/2666768112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b9370 msgr2=0x55d70d3c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba6ffd640 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b9370 0x55d70d3c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba6ffd640 1 -- 172.21.15.103:0/2666768112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 msgr2=0x55d70d3c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba6ffd640 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba6ffd640 1 -- 172.21.15.103:0/2666768112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70d3c43e0 con 0x55d70d3b42e0 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba7fff640 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b9370 0x55d70d3c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.046+0000 7ffba77fe640 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffba6ffd640 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b42e0 0x55d70d3c2dc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55d70d3afe90 tx=0x7ffb9c0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e2c1d8c8c076f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffb86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffb86ffd640 1 -- 172.21.15.103:0/2666768112 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c025030 con 0x55d70d3b42e0 2024-09-05T02:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffb86ffd640 1 -- 172.21.15.103:0/2666768112 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffb9c004680 con 0x55d70d3b42e0 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/2666768112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffba003c070 con 0x55d70d3b42e0 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffb86ffd640 1 -- 172.21.15.103:0/2666768112 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffb9c004880 con 0x55d70d3b42e0 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffbb0fcb740 10 client.15711 fetch_fsmap learned FSMap version 33 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/2666768112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d70d434640 con 0x55d70d3b42e0 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffb86ffd640 1 -- 172.21.15.103:0/2666768112 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba003c070 con 0x55d70d3b42e0 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffb86ffd640 1 -- 172.21.15.103:0/2666768112 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7ffb9c00f9f0 con 0x55d70d3b42e0 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffbb0fcb740 10 client.15711 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffbb0fcb740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffbb0fcb740 20 client.15711 populate_metadata read hostname 'smithi103' 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d70d434e00 0x55d70d437220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffbb0fcb740 4 client.15711 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.047+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/2666768112 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d70d31ed70 con 0x55d70d434e00 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.048+0000 7ffba77fe640 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d70d434e00 0x55d70d437220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:04:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.048+0000 7ffb86ffd640 10 client.15711 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:04:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:04:53.048+0000 7ffba77fe640 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d70d434e00 0x55d70d437220 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc25bb2f51b4cc20 server_cookie=fd97a3af669bf57d in_seq=0 out_seq=0 2024-09-05T02:04:53.874 INFO:tasks.workunit.client.0.smithi103.stdout: 18 5027 56 2942 137 2256 83 2024-09-05T02:04:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:54 smithi167 ceph-mon[35122]: pgmap v424: 129 pgs: 129 active+clean; 89 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.7 MiB/s wr, 348 op/s 2024-09-05T02:04:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:54 smithi139 ceph-mon[34262]: pgmap v424: 129 pgs: 129 active+clean; 89 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.7 MiB/s wr, 348 op/s 2024-09-05T02:04:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:54 smithi103 ceph-mon[30902]: pgmap v424: 129 pgs: 129 active+clean; 89 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.7 MiB/s wr, 348 op/s 2024-09-05T02:04:56.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:56 smithi167 ceph-mon[35122]: pgmap v425: 129 pgs: 129 active+clean; 90 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.6 MiB/s wr, 333 op/s 2024-09-05T02:04:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:56 smithi139 ceph-mon[34262]: pgmap v425: 129 pgs: 129 active+clean; 90 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.6 MiB/s wr, 333 op/s 2024-09-05T02:04:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:56 smithi103 ceph-mon[30902]: pgmap v425: 129 pgs: 129 active+clean; 90 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.6 MiB/s wr, 333 op/s 2024-09-05T02:04:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:04:58 smithi167 ceph-mon[35122]: pgmap v426: 129 pgs: 129 active+clean; 90 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.3 MiB/s wr, 284 op/s 2024-09-05T02:04:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:04:58 smithi139 ceph-mon[34262]: pgmap v426: 129 pgs: 129 active+clean; 90 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.3 MiB/s wr, 284 op/s 2024-09-05T02:04:58.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:04:58 smithi103 ceph-mon[30902]: pgmap v426: 129 pgs: 129 active+clean; 90 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.3 MiB/s wr, 284 op/s 2024-09-05T02:05:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:00 smithi167 ceph-mon[35122]: pgmap v427: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.1 MiB/s wr, 272 op/s 2024-09-05T02:05:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:00 smithi139 ceph-mon[34262]: pgmap v427: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.1 MiB/s wr, 272 op/s 2024-09-05T02:05:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:00 smithi103 ceph-mon[30902]: pgmap v427: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.1 MiB/s wr, 272 op/s 2024-09-05T02:05:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:02 smithi167 ceph-mon[35122]: pgmap v428: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 729 KiB/s wr, 224 op/s 2024-09-05T02:05:02.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:02 smithi139 ceph-mon[34262]: pgmap v428: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 729 KiB/s wr, 224 op/s 2024-09-05T02:05:02.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:02 smithi103 ceph-mon[30902]: pgmap v428: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 729 KiB/s wr, 224 op/s 2024-09-05T02:05:03.874 INFO:tasks.workunit.client.0.smithi103.stdout: 18 0 0 0 0 0 0 2024-09-05T02:05:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:04 smithi167 ceph-mon[35122]: pgmap v429: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-05T02:05:04.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:04 smithi139 ceph-mon[34262]: pgmap v429: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-05T02:05:04.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:04 smithi103 ceph-mon[30902]: pgmap v429: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-05T02:05:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:06 smithi167 ceph-mon[35122]: pgmap v430: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-09-05T02:05:06.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:06 smithi139 ceph-mon[34262]: pgmap v430: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-09-05T02:05:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:06 smithi103 ceph-mon[30902]: pgmap v430: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-09-05T02:05:07.121 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:05:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:05:07.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 -- 172.21.15.103:0/3262999339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0102790 msgr2=0x7f9cd01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:07.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3262999339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0102790 0x7f9cd01067e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009900 tx=0x7f9cc402a410 comp rx=0 tx=0).stop 2024-09-05T02:05:07.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 -- 172.21.15.103:0/3262999339 shutdown_connections 2024-09-05T02:05:07.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3262999339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd0106f10 0x7f9cd010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:07.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3262999339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0102790 0x7f9cd01067e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3262999339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd0101de0 0x7f9cd01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 -- 172.21.15.103:0/3262999339 >> 172.21.15.103:0/3262999339 conn(0x7f9cd00fd8c0 msgr2=0x7f9cd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 -- 172.21.15.103:0/3262999339 shutdown_connections 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 -- 172.21.15.103:0/3262999339 wait complete. 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.936+0000 7f9cd816c640 1 Processor -- start 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd816c640 1 -- start start 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd816c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd0101de0 0x7f9cd01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd816c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd0102790 0x7f9cd0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd816c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106f10 0x7f9cd019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd816c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd0111e40 con 0x7f9cd0106f10 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd816c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd0111cc0 con 0x7f9cd0102790 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd816c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd0111fc0 con 0x7f9cd0101de0 2024-09-05T02:05:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd6709640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106f10 0x7f9cd019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd6709640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106f10 0x7f9cd019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49240/0 (socket says 172.21.15.103:49240) 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd6709640 1 -- 172.21.15.103:0/3436960570 learned_addr learned my addr 172.21.15.103:0/3436960570 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd5707640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd0102790 0x7f9cd0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd5f08640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd0101de0 0x7f9cd01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd6709640 1 -- 172.21.15.103:0/3436960570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd0101de0 msgr2=0x7f9cd01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd6709640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd0101de0 0x7f9cd01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd6709640 1 -- 172.21.15.103:0/3436960570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd0102790 msgr2=0x7f9cd0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd6709640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd0102790 0x7f9cd0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd6709640 1 -- 172.21.15.103:0/3436960570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd019d370 con 0x7f9cd0106f10 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd5f08640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd0101de0 0x7f9cd01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cd6709640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106f10 0x7f9cd019cc90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800b970 tx=0x7f9cc800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.937+0000 7f9cc2ffd640 1 -- 172.21.15.103:0/3436960570 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8019070 con 0x7f9cd0106f10 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.938+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd019d660 con 0x7f9cd0106f10 2024-09-05T02:05:07.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.938+0000 7f9cc2ffd640 1 -- 172.21.15.103:0/3436960570 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cc8014440 con 0x7f9cd0106f10 2024-09-05T02:05:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.938+0000 7f9cc2ffd640 1 -- 172.21.15.103:0/3436960570 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80075d0 con 0x7f9cd0106f10 2024-09-05T02:05:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.938+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd01a4f40 con 0x7f9cd0106f10 2024-09-05T02:05:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.939+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd01021c0 con 0x7f9cd0106f10 2024-09-05T02:05:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.940+0000 7f9cc2ffd640 1 -- 172.21.15.103:0/3436960570 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9cc801c070 con 0x7f9cd0106f10 2024-09-05T02:05:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.940+0000 7f9cc2ffd640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cbc0a2b80 0x7f9cbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.941+0000 7f9cd5f08640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cbc0a2b80 0x7f9cbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.941+0000 7f9cc2ffd640 1 -- 172.21.15.103:0/3436960570 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9cc80ceed0 con 0x7f9cd0106f10 2024-09-05T02:05:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.941+0000 7f9cd5f08640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cbc0a2b80 0x7f9cbc0a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009b40 tx=0x7f9ccc005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:07.944+0000 7f9cc2ffd640 1 -- 172.21.15.103:0/3436960570 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9cc809ba80 con 0x7f9cd0106f10 2024-09-05T02:05:08.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.142+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cd0199720 con 0x7f9cd0106f10 2024-09-05T02:05:08.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.142+0000 7f9cc2ffd640 1 -- 172.21.15.103:0/3436960570 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12012 (secure 0 0 0) 0x7f9cd0199720 con 0x7f9cd0106f10 2024-09-05T02:05:08.144 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:05:08.144 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":33,"btime":"2024-09-05T02:04:34:292690+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:04:33.568912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:05:08.145 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 33 2024-09-05T02:05:08.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cbc0a2b80 msgr2=0x7f9cbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:08.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cbc0a2b80 0x7f9cbc0a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009b40 tx=0x7f9ccc005e50 comp rx=0 tx=0).stop 2024-09-05T02:05:08.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106f10 msgr2=0x7f9cd019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:08.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106f10 0x7f9cd019cc90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800b970 tx=0x7f9cc800be30 comp rx=0 tx=0).stop 2024-09-05T02:05:08.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 shutdown_connections 2024-09-05T02:05:08.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106f10 0x7f9cd019cc90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:08.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd0102790 0x7f9cd0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:08.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f9cbc0a2b80 0x7f9cbc0a5040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:08.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 --2- 172.21.15.103:0/3436960570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd0101de0 0x7f9cd01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:08.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.145+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 >> 172.21.15.103:0/3436960570 conn(0x7f9cd00fd8c0 msgr2=0x7f9cd00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:08.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.146+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 shutdown_connections 2024-09-05T02:05:08.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:08.146+0000 7f9cd816c640 1 -- 172.21.15.103:0/3436960570 wait complete. 2024-09-05T02:05:08.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:08 smithi103 ceph-mon[30902]: pgmap v431: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-05T02:05:08.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:08 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3436960570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:05:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:08 smithi167 ceph-mon[35122]: pgmap v431: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-05T02:05:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:08 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3436960570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:05:08.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:08 smithi139 ceph-mon[34262]: pgmap v431: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-05T02:05:08.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:08 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3436960570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:05:08.941 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id bd6d4922 --release --await' 2024-09-05T02:05:08.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id bd6d4922 --release --await 2024-09-05T02:05:09.188 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:05:09.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.694+0000 7fe2293cc640 1 -- 172.21.15.103:0/2137336560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241082a0 msgr2=0x7fe22410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.694+0000 7fe2293cc640 1 --2- 172.21.15.103:0/2137336560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241082a0 0x7fe22410ec40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009930 tx=0x7fe21c02a440 comp rx=0 tx=0).stop 2024-09-05T02:05:09.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 -- 172.21.15.103:0/2137336560 shutdown_connections 2024-09-05T02:05:09.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 --2- 172.21.15.103:0/2137336560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241082a0 0x7fe22410ec40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 --2- 172.21.15.103:0/2137336560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2241036e0 0x7fe224107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 --2- 172.21.15.103:0/2137336560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe224102d30 0x7fe224103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 -- 172.21.15.103:0/2137336560 >> 172.21.15.103:0/2137336560 conn(0x7fe2240fc7d0 msgr2=0x7fe2240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:09.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 -- 172.21.15.103:0/2137336560 shutdown_connections 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 -- 172.21.15.103:0/2137336560 wait complete. 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 Processor -- start 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 -- start start 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.695+0000 7fe2293cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe224102d30 0x7fe2241985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2293cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241036e0 0x7fe224198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2293cc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2241082a0 0x7fe22419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2293cc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe224112010 con 0x7fe2241036e0 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2293cc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe224111e90 con 0x7fe224102d30 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2293cc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe224112190 con 0x7fe2241082a0 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241036e0 0x7fe224198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241036e0 0x7fe224198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52000/0 (socket says 172.21.15.103:52000) 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2227fc640 1 -- 172.21.15.103:0/1126123318 learned_addr learned my addr 172.21.15.103:0/1126123318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe222ffd640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe224102d30 0x7fe2241985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2237fe640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2241082a0 0x7fe22419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2227fc640 1 -- 172.21.15.103:0/1126123318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2241082a0 msgr2=0x7fe22419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2227fc640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2241082a0 0x7fe22419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2227fc640 1 -- 172.21.15.103:0/1126123318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe224102d30 msgr2=0x7fe2241985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2227fc640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe224102d30 0x7fe2241985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2227fc640 1 -- 172.21.15.103:0/1126123318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22419d580 con 0x7fe2241036e0 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2237fe640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2241082a0 0x7fe22419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2227fc640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241036e0 0x7fe224198b10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006fd0 tx=0x7fe20c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe21bfff640 1 -- 172.21.15.103:0/1126123318 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c011830 con 0x7fe2241036e0 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe21bfff640 1 -- 172.21.15.103:0/1126123318 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe20c016070 con 0x7fe2241036e0 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2293cc640 1 -- 172.21.15.103:0/1126123318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22419d870 con 0x7fe2241036e0 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe21bfff640 1 -- 172.21.15.103:0/1126123318 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c010df0 con 0x7fe2241036e0 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.696+0000 7fe2293cc640 1 -- 172.21.15.103:0/1126123318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2241a5150 con 0x7fe2241036e0 2024-09-05T02:05:09.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.699+0000 7fe21bfff640 1 -- 172.21.15.103:0/1126123318 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe20c0119d0 con 0x7fe2241036e0 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.699+0000 7fe22b7a5740 1 Processor -- start 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.699+0000 7fe22b7a5740 1 -- start start 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.699+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 0x56525c71e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.699+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c728d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.699+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c72b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.699+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56525c683620 con 0x56525c71e980 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.699+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56525c62ae50 con 0x56525c729240 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.699+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56525c67f5e0 con 0x56525c71e060 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe21bfff640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe2100a2b80 0x7fe2100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c728d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c728d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52012/0 (socket says 172.21.15.103:52012) 2024-09-05T02:05:09.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe222ffd640 1 -- 172.21.15.103:0/2088869026 learned_addr learned my addr 172.21.15.103:0/2088869026 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe2237fe640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe2100a2b80 0x7fe2100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe21bfff640 1 -- 172.21.15.103:0/1126123318 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe20c0cea50 con 0x7fe2241036e0 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe2227fc640 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c72b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe222ffd640 1 -- 172.21.15.103:0/2088869026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 msgr2=0x56525c71e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe2237fe640 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 0x56525c71e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe222ffd640 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 0x56525c71e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe222ffd640 1 -- 172.21.15.103:0/2088869026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 msgr2=0x56525c72b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe222ffd640 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c72b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe222ffd640 1 -- 172.21.15.103:0/2088869026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525c61bd40 con 0x56525c71e980 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe2237fe640 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 0x56525c71e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe2227fc640 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c72b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe222ffd640 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c728d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe21400a4b0 tx=0x7fe2140096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f288721d6dd9dff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:09.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe2197fa640 1 -- 172.21.15.103:0/2088869026 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214018020 con 0x56525c71e980 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe2197fa640 1 -- 172.21.15.103:0/2088869026 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe214014cc0 con 0x56525c71e980 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe2237fe640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe2100a2b80 0x7fe2100a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005ec0 tx=0x7fe21c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.700+0000 7fe2197fa640 1 -- 172.21.15.103:0/2088869026 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214013cd0 con 0x56525c71e980 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 -- 172.21.15.103:0/2088869026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 msgr2=0x56525c728d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c728d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe21400a4b0 tx=0x7fe2140096d0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 -- 172.21.15.103:0/2088869026 shutdown_connections 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c72b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c728d00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/2088869026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 0x56525c71e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 -- 172.21.15.103:0/2088869026 >> 172.21.15.103:0/2088869026 conn(0x56525c71a5f0 msgr2=0x56525c71a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 -- 172.21.15.103:0/2088869026 shutdown_connections 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 -- 172.21.15.103:0/2088869026 wait complete. 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.701+0000 7fe22b7a5740 1 -- start start 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe22b7a5740 1 Processor -- start 2024-09-05T02:05:09.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 0x56525c726610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c726b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56525c72be10 con 0x56525c71e060 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56525c60bf10 con 0x56525c729240 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56525c72bf90 con 0x56525c71e980 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe22b7a5740 10 client.0 _ll_register_callbacks cb 0x56525c62fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7260d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52022/0 (socket says 172.21.15.103:52022) 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe222ffd640 1 -- 172.21.15.103:0/1842330555 learned_addr learned my addr 172.21.15.103:0/1842330555 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe2237fe640 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c726b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe222ffd640 1 -- 172.21.15.103:0/1842330555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 msgr2=0x56525c726610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe222ffd640 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 0x56525c726610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe222ffd640 1 -- 172.21.15.103:0/1842330555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 msgr2=0x56525c726b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe222ffd640 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c726b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe222ffd640 1 -- 172.21.15.103:0/1842330555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525c392580 con 0x56525c71e060 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe2237fe640 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c726b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe222ffd640 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7260d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe214004a80 tx=0x7fe214009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af0819cdbd92bab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.702+0000 7fe20a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1842330555 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214014a20 con 0x56525c71e060 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1842330555 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe214029480 con 0x56525c71e060 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1842330555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe214014cc0 con 0x56525c71e060 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1842330555 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe214029680 con 0x56525c71e060 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe22b7a5740 10 client.15729 fetch_fsmap learned FSMap version 33 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1842330555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56525c45c960 con 0x56525c71e060 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1842330555 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe214014cc0 con 0x56525c71e060 2024-09-05T02:05:09.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1842330555 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fe21401d550 con 0x56525c71e060 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe22b7a5740 10 client.15729 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe22b7a5740 10 client.15729 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe22b7a5740 20 client.15729 populate_metadata read hostname 'smithi103' 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56525c738cb0 0x56525c73b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe22b7a5740 4 client.15729 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1842330555 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56525c690b90 con 0x56525c738cb0 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.703+0000 7fe2227fc640 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56525c738cb0 0x56525c73b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.704+0000 7fe2227fc640 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56525c738cb0 0x56525c73b0a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa015c8df6c53da0 server_cookie=b5332b8b10157e3c in_seq=0 out_seq=0 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.704+0000 7fe20a7fc640 10 client.15729 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:09.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1842330555 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56525c690b90 con 0x56525c738cb0 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe20a7fc640 10 client.15729 handle_command_reply: tid=0 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 client.15729 shutdown 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1842330555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 msgr2=0x56525c7260d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7260d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe214004a80 tx=0x7fe214009b80 comp rx=0 tx=0).stop 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1842330555 shutdown_connections 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56525c738cb0 0x56525c73b0a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c726b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 0x56525c726610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe20a7fc640 0 client.15729 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1842330555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7260d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1842330555 >> 172.21.15.103:0/1842330555 conn(0x56525c71a5f0 msgr2=0x56525c7202d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1842330555 shutdown_connections 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1842330555 wait complete. 2024-09-05T02:05:09.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.705+0000 7fe22b7a5740 20 client.15729 trim_cache size 0 max 16384 2024-09-05T02:05:09.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.722+0000 7fe22b7a5740 1 Processor -- start 2024-09-05T02:05:09.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.722+0000 7fe22b7a5740 1 -- start start 2024-09-05T02:05:09.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.722+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7407c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.722+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 0x56525c740d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.722+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c723330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.722+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241aee50 con 0x56525c71e060 2024-09-05T02:05:09.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.722+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241b0b10 con 0x56525c729240 2024-09-05T02:05:09.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.722+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241af520 con 0x56525c71e980 2024-09-05T02:05:09.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7407c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52038/0 (socket says 172.21.15.103:52038) 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe222ffd640 1 -- 172.21.15.103:0/191810533 learned_addr learned my addr 172.21.15.103:0/191810533 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe2227fc640 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 0x56525c740d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe2237fe640 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c723330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe222ffd640 1 -- 172.21.15.103:0/191810533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 msgr2=0x56525c740d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe222ffd640 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 0x56525c740d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe222ffd640 1 -- 172.21.15.103:0/191810533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 msgr2=0x56525c723330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe222ffd640 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c723330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe222ffd640 1 -- 172.21.15.103:0/191810533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525c71a420 con 0x56525c71e060 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe2227fc640 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 0x56525c740d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe222ffd640 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7407c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe214017040 tx=0x7fe21400a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63dc6fb52ae397ff server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe218ff9640 1 -- 172.21.15.103:0/191810533 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214018070 con 0x56525c71e060 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe218ff9640 1 -- 172.21.15.103:0/191810533 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe214013640 con 0x56525c71e060 2024-09-05T02:05:09.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.723+0000 7fe218ff9640 1 -- 172.21.15.103:0/191810533 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214014a80 con 0x56525c71e060 2024-09-05T02:05:09.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 -- 172.21.15.103:0/191810533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 msgr2=0x56525c7407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7407c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe214017040 tx=0x7fe21400a400 comp rx=0 tx=0).stop 2024-09-05T02:05:09.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 -- 172.21.15.103:0/191810533 shutdown_connections 2024-09-05T02:05:09.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c723330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e980 0x56525c740d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/191810533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e060 0x56525c7407c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 -- 172.21.15.103:0/191810533 >> 172.21.15.103:0/191810533 conn(0x56525c71a5f0 msgr2=0x56525c7249a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:09.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 -- 172.21.15.103:0/191810533 shutdown_connections 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 -- 172.21.15.103:0/191810533 wait complete. 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.724+0000 7fe22b7a5740 1 -- start start 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe22b7a5740 1 Processor -- start 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 0x56525c72d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c72d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe22b7a5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c72dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241b0380 con 0x56525c71e980 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241afbf0 con 0x56525c729240 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe22b7a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241ace30 con 0x56525c71e060 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe22b7a5740 10 client.0 _ll_register_callbacks cb 0x7fe214013e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c72d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c72d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52046/0 (socket says 172.21.15.103:52046) 2024-09-05T02:05:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2227fc640 1 -- 172.21.15.103:0/1787266320 learned_addr learned my addr 172.21.15.103:0/1787266320 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe222ffd640 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 0x56525c72d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2237fe640 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c72dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2227fc640 1 -- 172.21.15.103:0/1787266320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 msgr2=0x56525c72d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2227fc640 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 0x56525c72d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2227fc640 1 -- 172.21.15.103:0/1787266320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 msgr2=0x56525c72dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2227fc640 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c72dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2227fc640 1 -- 172.21.15.103:0/1787266320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525c7a2990 con 0x56525c71e980 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe2227fc640 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c72d710 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00bde0 tx=0x7fe20c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e189dda0be63a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.725+0000 7fe20a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1787266320 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0b3070 con 0x56525c71e980 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1787266320 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe20c099200 con 0x56525c71e980 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1787266320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe214013640 con 0x56525c71e980 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1787266320 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe20c09f2b0 con 0x56525c71e980 2024-09-05T02:05:09.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe22b7a5740 10 client.15735 fetch_fsmap learned FSMap version 33 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1787266320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56525c7a3cf0 con 0x56525c71e980 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1787266320 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe214013640 con 0x56525c71e980 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1787266320 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fe20c09a350 con 0x56525c71e980 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe22b7a5740 10 client.15735 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe22b7a5740 10 client.15735 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe22b7a5740 20 client.15735 populate_metadata read hostname 'smithi103' 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56525c7a4510 0x56525c7a6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe22b7a5740 4 client.15735 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bd6d4922", "release": true, "await": true}] 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1787266320 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bd6d4922", "release": true, "await": true}) -- 0x56525c68b0a0 con 0x56525c7a4510 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.726+0000 7fe222ffd640 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56525c7a4510 0x56525c7a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.727+0000 7fe20a7fc640 10 client.15735 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:09.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.727+0000 7fe222ffd640 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56525c7a4510 0x56525c7a6930 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81841bcc8a6bff02 server_cookie=85c764b7342f16aa in_seq=0 out_seq=0 2024-09-05T02:05:09.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe20a7fc640 1 -- 172.21.15.103:0/1787266320 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56525c68b0a0 con 0x56525c7a4510 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe20a7fc640 10 client.15735 handle_command_reply: tid=0 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe22b7a5740 1 client.15735 shutdown 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1787266320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 msgr2=0x56525c72d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c72d710 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00bde0 tx=0x7fe20c00bf90 comp rx=0 tx=0).stop 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1787266320 shutdown_connections 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56525c7a4510 0x56525c7a6930 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56525c729240 0x56525c72dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c71e980 0x56525c72d710 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe22b7a5740 1 --2- 172.21.15.103:0/1787266320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56525c71e060 0x56525c72d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1787266320 >> 172.21.15.103:0/1787266320 conn(0x56525c71a5f0 msgr2=0x56525c718f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.822+0000 7fe20a7fc640 0 client.15735 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1787266320 shutdown_connections 2024-09-05T02:05:09.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe22b7a5740 1 -- 172.21.15.103:0/1787266320 wait complete. 2024-09-05T02:05:09.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe22b7a5740 20 client.15735 trim_cache size 0 max 16384 2024-09-05T02:05:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:05:09.826 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 33, 2024-09-05T02:05:09.826 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:05:09.826 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 8, 2024-09-05T02:05:09.826 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:05:09.826 INFO:teuthology.orchestra.run.smithi103.stdout: "bd6d4922": { 2024-09-05T02:05:09.826 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 8, 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 144.2, 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:05:09.827 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:05:09.828 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:05:09.828 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:05:09.828 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:05:09.828 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:05:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:05:09.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 -- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe2100a2b80 msgr2=0x7fe2100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe2100a2b80 0x7fe2100a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005ec0 tx=0x7fe21c031400 comp rx=0 tx=0).stop 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 -- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241036e0 msgr2=0x7fe224198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241036e0 0x7fe224198b10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006fd0 tx=0x7fe20c00b8e0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 -- 172.21.15.103:0/1126123318 shutdown_connections 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe2100a2b80 0x7fe2100a5040 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2241082a0 0x7fe22419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2241036e0 0x7fe224198b10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 --2- 172.21.15.103:0/1126123318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe224102d30 0x7fe2241985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.823+0000 7fe2293cc640 1 -- 172.21.15.103:0/1126123318 >> 172.21.15.103:0/1126123318 conn(0x7fe2240fc7d0 msgr2=0x7fe224105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.824+0000 7fe2293cc640 1 -- 172.21.15.103:0/1126123318 shutdown_connections 2024-09-05T02:05:09.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:09.824+0000 7fe2293cc640 1 -- 172.21.15.103:0/1126123318 wait complete. 2024-09-05T02:05:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.854+0000 7ffb86ffd640 1 -- 172.21.15.103:0/2666768112 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d70d31ed70 con 0x55d70d434e00 2024-09-05T02:05:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.854+0000 7ffb86ffd640 10 client.15711 handle_command_reply: tid=0 2024-09-05T02:05:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 client.15711 shutdown 2024-09-05T02:05:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/2666768112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b42e0 msgr2=0x55d70d3c2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b42e0 0x55d70d3c2dc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55d70d3afe90 tx=0x7ffb9c0052e0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/2666768112 shutdown_connections 2024-09-05T02:05:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d70d434e00 0x55d70d437220 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70d3b9370 0x55d70d3c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d70d3b42e0 0x55d70d3c2dc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffb86ffd640 0 client.15711 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 --2- 172.21.15.103:0/2666768112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d70d3b39a0 0x55d70d3c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/2666768112 >> 172.21.15.103:0/2666768112 conn(0x55d70d3af010 msgr2=0x55d70d3ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/2666768112 shutdown_connections 2024-09-05T02:05:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 1 -- 172.21.15.103:0/2666768112 wait complete. 2024-09-05T02:05:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.855+0000 7ffbb0fcb740 20 client.15711 trim_cache size 0 max 16384 2024-09-05T02:05:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:05:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:05:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:05:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:05:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 -- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffb900a2c30 msgr2=0x7ffb900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffb900a2c30 0x7ffb900a50f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffba0005fd0 tx=0x7ffba000ad30 comp rx=0 tx=0).stop 2024-09-05T02:05:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 -- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 msgr2=0x7ffba8075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 0x7ffba8075fa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009230 tx=0x7ffb98004340 comp rx=0 tx=0).stop 2024-09-05T02:05:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 -- 172.21.15.103:0/1200498088 shutdown_connections 2024-09-05T02:05:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ffb900a2c30 0x7ffb900a50f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffba8103710 0x7ffba8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffba8102d50 0x7ffba80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 --2- 172.21.15.103:0/1200498088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba8101b60 0x7ffba8075fa0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 -- 172.21.15.103:0/1200498088 >> 172.21.15.103:0/1200498088 conn(0x7ffba80fd2e0 msgr2=0x7ffba8105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.856+0000 7ffbae22e640 1 -- 172.21.15.103:0/1200498088 shutdown_connections 2024-09-05T02:05:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:09.857+0000 7ffbae22e640 1 -- 172.21.15.103:0/1200498088 wait complete. 2024-09-05T02:05:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:10 smithi167 ceph-mon[35122]: pgmap v432: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-05T02:05:10.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:10 smithi139 ceph-mon[34262]: pgmap v432: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-05T02:05:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:10 smithi103 ceph-mon[30902]: pgmap v432: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-05T02:05:10.720 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bd6d4922', total seconds elapsed: 27.8 2024-09-05T02:05:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:12 smithi167 ceph-mon[35122]: pgmap v433: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-05T02:05:12.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:12 smithi139 ceph-mon[34262]: pgmap v433: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-05T02:05:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:12 smithi103 ceph-mon[30902]: pgmap v433: 129 pgs: 129 active+clean; 90 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-05T02:05:13.874 INFO:tasks.workunit.client.0.smithi103.stdout: 19 3696 130 2057 115 1585 141 2024-09-05T02:05:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:14 smithi167 ceph-mon[35122]: pgmap v434: 129 pgs: 129 active+clean; 91 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 68 KiB/s wr, 16 op/s 2024-09-05T02:05:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:14 smithi139 ceph-mon[34262]: pgmap v434: 129 pgs: 129 active+clean; 91 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 68 KiB/s wr, 16 op/s 2024-09-05T02:05:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:14 smithi103 ceph-mon[30902]: pgmap v434: 129 pgs: 129 active+clean; 91 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 68 KiB/s wr, 16 op/s 2024-09-05T02:05:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:16 smithi167 ceph-mon[35122]: pgmap v435: 129 pgs: 129 active+clean; 102 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-05T02:05:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:16 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:05:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:16 smithi139 ceph-mon[34262]: pgmap v435: 129 pgs: 129 active+clean; 102 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-05T02:05:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:16 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:05:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:16 smithi103 ceph-mon[30902]: pgmap v435: 129 pgs: 129 active+clean; 102 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-05T02:05:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:16 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:05:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:05:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:05:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:05:17.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:05:17.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:05:17.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:05:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:05:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:05:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:05:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:18 smithi167 ceph-mon[35122]: pgmap v436: 129 pgs: 129 active+clean; 103 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-05T02:05:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:18 smithi139 ceph-mon[34262]: pgmap v436: 129 pgs: 129 active+clean; 103 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-05T02:05:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:18 smithi103 ceph-mon[30902]: pgmap v436: 129 pgs: 129 active+clean; 103 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-05T02:05:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:20 smithi167 ceph-mon[35122]: pgmap v437: 129 pgs: 129 active+clean; 108 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-09-05T02:05:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:20 smithi139 ceph-mon[34262]: pgmap v437: 129 pgs: 129 active+clean; 108 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-09-05T02:05:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:20 smithi103 ceph-mon[30902]: pgmap v437: 129 pgs: 129 active+clean; 108 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-09-05T02:05:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:22 smithi167 ceph-mon[35122]: pgmap v438: 129 pgs: 129 active+clean; 113 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-09-05T02:05:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:22 smithi139 ceph-mon[34262]: pgmap v438: 129 pgs: 129 active+clean; 113 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-09-05T02:05:22.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:22 smithi103 ceph-mon[30902]: pgmap v438: 129 pgs: 129 active+clean; 113 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-09-05T02:05:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.984+0000 7f426ecc9640 1 -- 172.21.15.103:0/1555406678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268101b60 msgr2=0x7f4268101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.984+0000 7f426ecc9640 1 --2- 172.21.15.103:0/1555406678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268101b60 0x7f4268101f60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f425c009900 tx=0x7f425c02a410 comp rx=0 tx=0).stop 2024-09-05T02:05:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.984+0000 7f426ecc9640 1 -- 172.21.15.103:0/1555406678 shutdown_connections 2024-09-05T02:05:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.984+0000 7f426ecc9640 1 --2- 172.21.15.103:0/1555406678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4268103710 0x7f4268109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.984+0000 7f426ecc9640 1 --2- 172.21.15.103:0/1555406678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4268102d50 0x7f42681031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.984+0000 7f426ecc9640 1 --2- 172.21.15.103:0/1555406678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268101b60 0x7f4268101f60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.984+0000 7f426ecc9640 1 -- 172.21.15.103:0/1555406678 >> 172.21.15.103:0/1555406678 conn(0x7f42680fd2e0 msgr2=0x7f42680ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.984+0000 7f426ecc9640 1 -- 172.21.15.103:0/1555406678 shutdown_connections 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.984+0000 7f426ecc9640 1 -- 172.21.15.103:0/1555406678 wait complete. 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ecc9640 1 Processor -- start 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ecc9640 1 -- start start 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ecc9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4268101b60 0x7f4268199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ecc9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268102d50 0x7f426819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ecc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4268103710 0x7f42681a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ecc9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4268075780 con 0x7f4268102d50 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ecc9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4268075600 con 0x7f4268103710 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ecc9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4268075900 con 0x7f4268101b60 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f4267fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268102d50 0x7f426819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ca65640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4268101b60 0x7f4268199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ca65640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4268101b60 0x7f4268199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:49914/0 (socket says 172.21.15.103:49914) 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426ca65640 1 -- 172.21.15.103:0/486755651 learned_addr learned my addr 172.21.15.103:0/486755651 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f4267fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268102d50 0x7f426819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48728/0 (socket says 172.21.15.103:48728) 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.985+0000 7f426d266640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4268103710 0x7f42681a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f4267fff640 1 -- 172.21.15.103:0/486755651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4268101b60 msgr2=0x7f4268199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f4267fff640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4268101b60 0x7f4268199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f4267fff640 1 -- 172.21.15.103:0/486755651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4268103710 msgr2=0x7f42681a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f4267fff640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4268103710 0x7f42681a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f4267fff640 1 -- 172.21.15.103:0/486755651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42681a1b40 con 0x7f4268102d50 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f426d266640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4268103710 0x7f42681a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f4267fff640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268102d50 0x7f426819a460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f425400d6d0 tx=0x7f425400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f4265ffb640 1 -- 172.21.15.103:0/486755651 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254017070 con 0x7f4268102d50 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f4265ffb640 1 -- 172.21.15.103:0/486755651 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4254009820 con 0x7f4268102d50 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f426ecc9640 1 -- 172.21.15.103:0/486755651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42681a1e30 con 0x7f4268102d50 2024-09-05T02:05:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f4265ffb640 1 -- 172.21.15.103:0/486755651 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254004c40 con 0x7f4268102d50 2024-09-05T02:05:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.986+0000 7f426ecc9640 1 -- 172.21.15.103:0/486755651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42681a2340 con 0x7f4268102d50 2024-09-05T02:05:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.989+0000 7f4265ffb640 1 -- 172.21.15.103:0/486755651 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f42540099d0 con 0x7f4268102d50 2024-09-05T02:05:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f4271a60740 1 Processor -- start 2024-09-05T02:05:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f4271a60740 1 -- start start 2024-09-05T02:05:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f4271a60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b5dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f4265ffb640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f42580a2cb0 0x7f42580a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f4271a60740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5e2e0 0x560425b62da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f426d266640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b5dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f4271a60740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 0x560425b65830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f4271a60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560425ac12f0 con 0x560425b5d9a0 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f4271a60740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560425a69270 con 0x560425b63370 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f4271a60740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560425abd2b0 con 0x560425b5e2e0 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f426d266640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b5dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48734/0 (socket says 172.21.15.103:48734) 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f426d266640 1 -- 172.21.15.103:0/600997378 learned_addr learned my addr 172.21.15.103:0/600997378 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.990+0000 7f426ca65640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f42580a2cb0 0x7f42580a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f426d266640 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 0x560425b65830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f426d266640 1 -- 172.21.15.103:0/600997378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5e2e0 msgr2=0x560425b62da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f4267fff640 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5e2e0 0x560425b62da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f426d266640 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5e2e0 0x560425b62da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f426d266640 1 -- 172.21.15.103:0/600997378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 msgr2=0x560425b65830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f426d266640 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 0x560425b65830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f426d266640 1 -- 172.21.15.103:0/600997378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560425a68340 con 0x560425b5d9a0 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f4267fff640 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5e2e0 0x560425b62da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f4265ffb640 1 -- 172.21.15.103:0/486755651 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f42540cee70 con 0x7f4268102d50 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f426ca65640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f42580a2cb0 0x7f42580a5170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f425c0096c0 tx=0x7f425c005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.991+0000 7f426d266640 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b5dda0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f42600149f0 tx=0x7f4260009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c50fe8401a87309 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.992+0000 7f4252ffd640 1 -- 172.21.15.103:0/600997378 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426001a070 con 0x560425b5d9a0 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.992+0000 7f4252ffd640 1 -- 172.21.15.103:0/600997378 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4260010e40 con 0x560425b5d9a0 2024-09-05T02:05:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.992+0000 7f4252ffd640 1 -- 172.21.15.103:0/600997378 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260011d00 con 0x560425b5d9a0 2024-09-05T02:05:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 -- 172.21.15.103:0/600997378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 msgr2=0x560425b5dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b5dda0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f42600149f0 tx=0x7f4260009d00 comp rx=0 tx=0).stop 2024-09-05T02:05:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 -- 172.21.15.103:0/600997378 shutdown_connections 2024-09-05T02:05:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 0x560425b65830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5e2e0 0x560425b62da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 --2- 172.21.15.103:0/600997378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b5dda0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 -- 172.21.15.103:0/600997378 >> 172.21.15.103:0/600997378 conn(0x560425b59010 msgr2=0x560425b59410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 -- 172.21.15.103:0/600997378 shutdown_connections 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 -- 172.21.15.103:0/600997378 wait complete. 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.993+0000 7f4271a60740 1 -- start start 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4271a60740 1 Processor -- start 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4271a60740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5d9a0 0x560425b70470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4271a60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5e2e0 0x560425b709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4271a60740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 0x560425b77980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4271a60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560425b66340 con 0x560425b5e2e0 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4271a60740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560425a4a290 con 0x560425b63370 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4271a60740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560425b664c0 con 0x560425b5d9a0 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4271a60740 10 client.0 _ll_register_callbacks cb 0x560425a7b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:05:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4267fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5e2e0 0x560425b709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4267fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5e2e0 0x560425b709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48738/0 (socket says 172.21.15.103:48738) 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f4267fff640 1 -- 172.21.15.103:0/4002019749 learned_addr learned my addr 172.21.15.103:0/4002019749 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.994+0000 7f426d266640 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 0x560425b77980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.995+0000 7f426ca65640 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5d9a0 0x560425b70470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.995+0000 7f4267fff640 1 -- 172.21.15.103:0/4002019749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5d9a0 msgr2=0x560425b70470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.995+0000 7f4267fff640 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5d9a0 0x560425b70470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.995+0000 7f4267fff640 1 -- 172.21.15.103:0/4002019749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 msgr2=0x560425b77980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.995+0000 7f4267fff640 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 0x560425b77980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.995+0000 7f4267fff640 1 -- 172.21.15.103:0/4002019749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604257d4210 con 0x560425b5e2e0 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.995+0000 7f426d266640 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 0x560425b77980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.995+0000 7f4267fff640 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5e2e0 0x560425b709b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4254097780 tx=0x7f4254018f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cac312ac70e0a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.995+0000 7f4247fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.996+0000 7f4247fff640 1 -- 172.21.15.103:0/4002019749 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254020040 con 0x560425b5e2e0 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.996+0000 7f4247fff640 1 -- 172.21.15.103:0/4002019749 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4254097990 con 0x560425b5e2e0 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.996+0000 7f4271a60740 1 -- 172.21.15.103:0/4002019749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4260010e40 con 0x560425b5e2e0 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.996+0000 7f4247fff640 1 -- 172.21.15.103:0/4002019749 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4254065530 con 0x560425b5e2e0 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4271a60740 10 client.15747 fetch_fsmap learned FSMap version 33 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4271a60740 1 -- 172.21.15.103:0/4002019749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5604258a0620 con 0x560425b5e2e0 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4247fff640 1 -- 172.21.15.103:0/4002019749 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4260010e40 con 0x560425b5e2e0 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4247fff640 1 -- 172.21.15.103:0/4002019749 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f42540650c0 con 0x560425b5e2e0 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4271a60740 10 client.15747 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4271a60740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4271a60740 20 client.15747 populate_metadata read hostname 'smithi103' 2024-09-05T02:05:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4271a60740 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560425b7c050 0x560425b7e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4271a60740 4 client.15747 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:05:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f4271a60740 1 -- 172.21.15.103:0/4002019749 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560425acea20 con 0x560425b7c050 2024-09-05T02:05:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.997+0000 7f426ca65640 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560425b7c050 0x560425b7e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.998+0000 7f426ca65640 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560425b7c050 0x560425b7e490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3a073dc87af1f46 server_cookie=bd88006e618b7b0b in_seq=0 out_seq=0 2024-09-05T02:05:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.998+0000 7f4247fff640 10 client.15747 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4247fff640 1 -- 172.21.15.103:0/4002019749 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560425acea20 con 0x560425b7c050 2024-09-05T02:05:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4247fff640 10 client.15747 handle_command_reply: tid=0 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 client.15747 shutdown 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 -- 172.21.15.103:0/4002019749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5e2e0 msgr2=0x560425b709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5e2e0 0x560425b709b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4254097780 tx=0x7f4254018f10 comp rx=0 tx=0).stop 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 -- 172.21.15.103:0/4002019749 shutdown_connections 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560425b7c050 0x560425b7e490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b63370 0x560425b77980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5e2e0 0x560425b709b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 --2- 172.21.15.103:0/4002019749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b5d9a0 0x560425b70470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4247fff640 0 client.15747 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 -- 172.21.15.103:0/4002019749 >> 172.21.15.103:0/4002019749 conn(0x560425b59010 msgr2=0x560425b5a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 -- 172.21.15.103:0/4002019749 shutdown_connections 2024-09-05T02:05:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 1 -- 172.21.15.103:0/4002019749 wait complete. 2024-09-05T02:05:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:22.999+0000 7f4271a60740 20 client.15747 trim_cache size 0 max 16384 2024-09-05T02:05:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.019+0000 7f4271a60740 1 Processor -- start 2024-09-05T02:05:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.019+0000 7f4271a60740 1 -- start start 2024-09-05T02:05:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.019+0000 7f4271a60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.019+0000 7f4271a60740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b77940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.019+0000 7f4271a60740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b6f5b0 0x560425b82d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.019+0000 7f4271a60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42681a8540 con 0x560425b5d9a0 2024-09-05T02:05:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.019+0000 7f4271a60740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42681aa200 con 0x560425b6f5b0 2024-09-05T02:05:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.019+0000 7f4271a60740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42681a8c10 con 0x560425b63370 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f4267fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b77940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f426ca65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f4267fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b77940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:49940/0 (socket says 172.21.15.103:49940) 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f4267fff640 1 -- 172.21.15.103:0/752620063 learned_addr learned my addr 172.21.15.103:0/752620063 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f426d266640 1 --2- 172.21.15.103:0/752620063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b6f5b0 0x560425b82d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f4267fff640 1 -- 172.21.15.103:0/752620063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b6f5b0 msgr2=0x560425b82d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f4267fff640 1 --2- 172.21.15.103:0/752620063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b6f5b0 0x560425b82d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f4267fff640 1 -- 172.21.15.103:0/752620063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 msgr2=0x560425b580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f4267fff640 1 --2- 172.21.15.103:0/752620063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f4267fff640 1 -- 172.21.15.103:0/752620063 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560425b58e30 con 0x560425b63370 2024-09-05T02:05:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f4267fff640 1 --2- 172.21.15.103:0/752620063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b77940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f425409f770 tx=0x7f4254099830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7c8a6db4d76448 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f42527fc640 1 -- 172.21.15.103:0/752620063 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254014070 con 0x560425b63370 2024-09-05T02:05:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f42527fc640 1 -- 172.21.15.103:0/752620063 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4254009d40 con 0x560425b63370 2024-09-05T02:05:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.020+0000 7f42527fc640 1 -- 172.21.15.103:0/752620063 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254020e90 con 0x560425b63370 2024-09-05T02:05:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.021+0000 7f4271a60740 1 -- 172.21.15.103:0/752620063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 msgr2=0x560425b77940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.021+0000 7f4271a60740 1 --2- 172.21.15.103:0/752620063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b77940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f425409f770 tx=0x7f4254099830 comp rx=0 tx=0).stop 2024-09-05T02:05:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.021+0000 7f4271a60740 1 -- 172.21.15.103:0/752620063 shutdown_connections 2024-09-05T02:05:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.021+0000 7f4271a60740 1 --2- 172.21.15.103:0/752620063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b6f5b0 0x560425b82d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.021+0000 7f4271a60740 1 --2- 172.21.15.103:0/752620063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b77940 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.021+0000 7f4271a60740 1 --2- 172.21.15.103:0/752620063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.021+0000 7f4271a60740 1 -- 172.21.15.103:0/752620063 >> 172.21.15.103:0/752620063 conn(0x560425b59010 msgr2=0x560425b579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.021+0000 7f4271a60740 1 -- 172.21.15.103:0/752620063 shutdown_connections 2024-09-05T02:05:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.021+0000 7f4271a60740 1 -- 172.21.15.103:0/752620063 wait complete. 2024-09-05T02:05:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.022+0000 7f4271a60740 1 -- start start 2024-09-05T02:05:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.022+0000 7f4271a60740 1 Processor -- start 2024-09-05T02:05:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.022+0000 7f4271a60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b75450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.022+0000 7f4271a60740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b75990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.022+0000 7f4271a60740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b76950 0x560425b76db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.022+0000 7f426ca65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b75450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f426ca65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b75450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48768/0 (socket says 172.21.15.103:48768) 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f426ca65640 1 -- 172.21.15.103:0/1417187323 learned_addr learned my addr 172.21.15.103:0/1417187323 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42681a9a70 con 0x560425b5d9a0 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42681a92e0 con 0x560425b76950 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42681a6520 con 0x560425b63370 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f4267fff640 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b75990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f4271a60740 10 client.0 _ll_register_callbacks cb 0x7f42540057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f426d266640 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b76950 0x560425b76db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f426ca65640 1 -- 172.21.15.103:0/1417187323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 msgr2=0x560425b75990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f426ca65640 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b75990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f426ca65640 1 -- 172.21.15.103:0/1417187323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b76950 msgr2=0x560425b76db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f426ca65640 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b76950 0x560425b76db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f426ca65640 1 -- 172.21.15.103:0/1417187323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560425b77340 con 0x560425b5d9a0 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f426ca65640 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b75450 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f425c009230 tx=0x7f425c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49497c9149fd86af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f4247fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f4247fff640 1 -- 172.21.15.103:0/1417187323 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c035050 con 0x560425b5d9a0 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f4247fff640 1 -- 172.21.15.103:0/1417187323 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f425c0037f0 con 0x560425b5d9a0 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.023+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560425ac8d70 con 0x560425b5d9a0 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4247fff640 1 -- 172.21.15.103:0/1417187323 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f425c0039f0 con 0x560425b5d9a0 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4271a60740 10 client.15753 fetch_fsmap learned FSMap version 33 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560425be75d0 con 0x560425b5d9a0 2024-09-05T02:05:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4247fff640 1 -- 172.21.15.103:0/1417187323 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560425ac8d70 con 0x560425b5d9a0 2024-09-05T02:05:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4247fff640 1 -- 172.21.15.103:0/1417187323 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f425c05b470 con 0x560425b5d9a0 2024-09-05T02:05:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4271a60740 10 client.15753 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:05:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4271a60740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:05:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4271a60740 20 client.15753 populate_metadata read hostname 'smithi103' 2024-09-05T02:05:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4271a60740 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560425be8020 0x560425bea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4271a60740 4 client.15753 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:05:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.024+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560425a45900 con 0x560425be8020 2024-09-05T02:05:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.025+0000 7f4267fff640 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560425be8020 0x560425bea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.026+0000 7f4267fff640 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560425be8020 0x560425bea440 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9339a1050c479f7 server_cookie=99a4c7c13bdc2b94 in_seq=0 out_seq=0 2024-09-05T02:05:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.026+0000 7f4247fff640 10 client.15753 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4247fff640 1 -- 172.21.15.103:0/1417187323 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560425a45900 con 0x560425be8020 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4247fff640 10 client.15753 handle_command_reply: tid=0 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4271a60740 1 client.15753 shutdown 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 msgr2=0x560425b75450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4271a60740 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b75450 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f425c009230 tx=0x7f425c02a410 comp rx=0 tx=0).stop 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 shutdown_connections 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4271a60740 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560425be8020 0x560425bea440 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4271a60740 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560425b76950 0x560425b76db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4271a60740 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560425b63370 0x560425b75990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4271a60740 1 --2- 172.21.15.103:0/1417187323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560425b5d9a0 0x560425b75450 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4247fff640 0 client.15753 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.094+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 >> 172.21.15.103:0/1417187323 conn(0x560425b59010 msgr2=0x560425b581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.095+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 shutdown_connections 2024-09-05T02:05:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.095+0000 7f4271a60740 1 -- 172.21.15.103:0/1417187323 wait complete. 2024-09-05T02:05:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.095+0000 7f4271a60740 20 client.15753 trim_cache size 0 max 16384 2024-09-05T02:05:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:05:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:05:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:05:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:05:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.095+0000 7f426ecc9640 1 -- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f42580a2cb0 msgr2=0x7f42580a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.095+0000 7f426ecc9640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f42580a2cb0 0x7f42580a5170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f425c0096c0 tx=0x7f425c005dd0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 -- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268102d50 msgr2=0x7f426819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268102d50 0x7f426819a460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f425400d6d0 tx=0x7f425400db90 comp rx=0 tx=0).stop 2024-09-05T02:05:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 -- 172.21.15.103:0/486755651 shutdown_connections 2024-09-05T02:05:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4268103710 0x7f42681a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4268102d50 0x7f426819a460 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f42580a2cb0 0x7f42580a5170 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 --2- 172.21.15.103:0/486755651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4268101b60 0x7f4268199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 -- 172.21.15.103:0/486755651 >> 172.21.15.103:0/486755651 conn(0x7f42680fd2e0 msgr2=0x7f42680fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 -- 172.21.15.103:0/486755651 shutdown_connections 2024-09-05T02:05:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:23.096+0000 7f426ecc9640 1 -- 172.21.15.103:0/486755651 wait complete. 2024-09-05T02:05:23.877 INFO:tasks.workunit.client.0.smithi103.stdout: 25 10336 310 6487 256 3790 179 2024-09-05T02:05:24.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:24 smithi167 ceph-mon[35122]: pgmap v439: 129 pgs: 129 active+clean; 119 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 261 op/s 2024-09-05T02:05:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:24 smithi139 ceph-mon[34262]: pgmap v439: 129 pgs: 129 active+clean; 119 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 261 op/s 2024-09-05T02:05:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:24 smithi103 ceph-mon[30902]: pgmap v439: 129 pgs: 129 active+clean; 119 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 261 op/s 2024-09-05T02:05:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:26 smithi167 ceph-mon[35122]: pgmap v440: 129 pgs: 129 active+clean; 124 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-05T02:05:26.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:26 smithi139 ceph-mon[34262]: pgmap v440: 129 pgs: 129 active+clean; 124 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-05T02:05:26.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:26 smithi103 ceph-mon[30902]: pgmap v440: 129 pgs: 129 active+clean; 124 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-05T02:05:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:28 smithi167 ceph-mon[35122]: pgmap v441: 129 pgs: 129 active+clean; 126 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 294 op/s 2024-09-05T02:05:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:28 smithi139 ceph-mon[34262]: pgmap v441: 129 pgs: 129 active+clean; 126 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 294 op/s 2024-09-05T02:05:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:28 smithi103 ceph-mon[30902]: pgmap v441: 129 pgs: 129 active+clean; 126 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 294 op/s 2024-09-05T02:05:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:30 smithi167 ceph-mon[35122]: pgmap v442: 129 pgs: 129 active+clean; 133 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-09-05T02:05:30.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:30 smithi139 ceph-mon[34262]: pgmap v442: 129 pgs: 129 active+clean; 133 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-09-05T02:05:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:30 smithi103 ceph-mon[30902]: pgmap v442: 129 pgs: 129 active+clean; 133 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-09-05T02:05:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:32 smithi167 ceph-mon[35122]: pgmap v443: 129 pgs: 129 active+clean; 141 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 345 op/s 2024-09-05T02:05:32.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:32 smithi139 ceph-mon[34262]: pgmap v443: 129 pgs: 129 active+clean; 141 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 345 op/s 2024-09-05T02:05:32.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:32 smithi103 ceph-mon[30902]: pgmap v443: 129 pgs: 129 active+clean; 141 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 345 op/s 2024-09-05T02:05:33.874 INFO:tasks.workunit.client.0.smithi103.stdout: 35 10426 548 6115 219 3177 337 2024-09-05T02:05:34.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:34 smithi103 ceph-mon[30902]: pgmap v444: 129 pgs: 129 active+clean; 148 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-09-05T02:05:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:34 smithi167 ceph-mon[35122]: pgmap v444: 129 pgs: 129 active+clean; 148 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-09-05T02:05:34.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:34 smithi139 ceph-mon[34262]: pgmap v444: 129 pgs: 129 active+clean; 148 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-09-05T02:05:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.221+0000 7f92fd427640 1 -- 172.21.15.103:0/2588101380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92f8101c40 msgr2=0x7f92f81020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.221+0000 7f92fd427640 1 --2- 172.21.15.103:0/2588101380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92f8101c40 0x7f92f81020c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009900 tx=0x7f92ec02a410 comp rx=0 tx=0).stop 2024-09-05T02:05:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.221+0000 7f92fd427640 1 -- 172.21.15.103:0/2588101380 shutdown_connections 2024-09-05T02:05:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.221+0000 7f92fd427640 1 --2- 172.21.15.103:0/2588101380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92f8102600 0x7f92f8108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.221+0000 7f92fd427640 1 --2- 172.21.15.103:0/2588101380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92f8101c40 0x7f92f81020c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.221+0000 7f92fd427640 1 --2- 172.21.15.103:0/2588101380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f8100a50 0x7f92f8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.221+0000 7f92fd427640 1 -- 172.21.15.103:0/2588101380 >> 172.21.15.103:0/2588101380 conn(0x7f92f80fc1b0 msgr2=0x7f92f80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.221+0000 7f92fd427640 1 -- 172.21.15.103:0/2588101380 shutdown_connections 2024-09-05T02:05:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.221+0000 7f92fd427640 1 -- 172.21.15.103:0/2588101380 wait complete. 2024-09-05T02:05:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92fd427640 1 Processor -- start 2024-09-05T02:05:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92fd427640 1 -- start start 2024-09-05T02:05:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92fd427640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f8100a50 0x7f92f810b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92fd427640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92f8101c40 0x7f92f8109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92fd427640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92f8102600 0x7f92f8109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92fd427640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92f810d6d0 con 0x7f92f8102600 2024-09-05T02:05:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92fd427640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f92f810d550 con 0x7f92f8101c40 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92fd427640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92f810d850 con 0x7f92f8100a50 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92f77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92f8102600 0x7f92f8109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.222+0000 7f92f6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f8100a50 0x7f92f810b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92f8102600 0x7f92f8109f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57558/0 (socket says 172.21.15.103:57558) 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f8100a50 0x7f92f810b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:47076/0 (socket says 172.21.15.103:47076) 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f77fe640 1 -- 172.21.15.103:0/1365482059 learned_addr learned my addr 172.21.15.103:0/1365482059 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f67fc640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92f8101c40 0x7f92f8109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f6ffd640 1 -- 172.21.15.103:0/1365482059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92f8101c40 msgr2=0x7f92f8109a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92f8101c40 0x7f92f8109a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f6ffd640 1 -- 172.21.15.103:0/1365482059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92f8102600 msgr2=0x7f92f8109f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92f8102600 0x7f92f8109f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f6ffd640 1 -- 172.21.15.103:0/1365482059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f810a7e0 con 0x7f92f8100a50 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f67fc640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92f8101c40 0x7f92f8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f8100a50 0x7f92f810b3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f92e800e9c0 tx=0x7f92e800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92dffff640 1 -- 172.21.15.103:0/1365482059 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e800cd80 con 0x7f92f8100a50 2024-09-05T02:05:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92fd427640 1 -- 172.21.15.103:0/1365482059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f81a1e10 con 0x7f92f8100a50 2024-09-05T02:05:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92dffff640 1 -- 172.21.15.103:0/1365482059 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92e80108e0 con 0x7f92f8100a50 2024-09-05T02:05:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92fd427640 1 -- 172.21.15.103:0/1365482059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f81a2270 con 0x7f92f8100a50 2024-09-05T02:05:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.223+0000 7f92dffff640 1 -- 172.21.15.103:0/1365482059 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8018860 con 0x7f92f8100a50 2024-09-05T02:05:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.227+0000 7f92dffff640 1 -- 172.21.15.103:0/1365482059 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f92e801f020 con 0x7f92f8100a50 2024-09-05T02:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.227+0000 7f93001c4740 1 Processor -- start 2024-09-05T02:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.227+0000 7f93001c4740 1 -- start start 2024-09-05T02:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.227+0000 7f93001c4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 0x55f5649cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.227+0000 7f93001c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.227+0000 7f93001c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.227+0000 7f93001c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f56492e2f0 con 0x55f5649d0370 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.227+0000 7f93001c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f5648d6270 con 0x55f5649cb2e0 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.227+0000 7f93001c4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f56492a2b0 con 0x55f5649ca9a0 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:44268/0 (socket says 172.21.15.103:44268) 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f6ffd640 1 -- 172.21.15.103:0/431686238 learned_addr learned my addr 172.21.15.103:0/431686238 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f77fe640 1 --2- 172.21.15.103:0/431686238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 0x55f5649cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f67fc640 1 -- 172.21.15.103:0/431686238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 msgr2=0x55f5649cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f67fc640 1 --2- 172.21.15.103:0/431686238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 0x55f5649cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f67fc640 1 -- 172.21.15.103:0/431686238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 msgr2=0x55f5649cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f67fc640 1 --2- 172.21.15.103:0/431686238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f67fc640 1 -- 172.21.15.103:0/431686238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5648d5340 con 0x55f5649d0370 2024-09-05T02:05:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.228+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/431686238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.229+0000 7f92f67fc640 1 --2- 172.21.15.103:0/431686238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649d2830 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0314e0 tx=0x7f92ec032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44ffc212d271e65 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.229+0000 7f92dffff640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92e00a2d30 0x7f92e00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.229+0000 7f92f77fe640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92e00a2d30 0x7f92e00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.229+0000 7f92dffff640 1 -- 172.21.15.103:0/1365482059 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f92e80cf2c0 con 0x7f92f8100a50 2024-09-05T02:05:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.229+0000 7f92dd7fa640 1 -- 172.21.15.103:0/431686238 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec035070 con 0x55f5649d0370 2024-09-05T02:05:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.229+0000 7f92dd7fa640 1 -- 172.21.15.103:0/431686238 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec031880 con 0x55f5649d0370 2024-09-05T02:05:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.229+0000 7f92dd7fa640 1 -- 172.21.15.103:0/431686238 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec003e20 con 0x55f5649d0370 2024-09-05T02:05:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 -- 172.21.15.103:0/431686238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 msgr2=0x55f5649d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f92f77fe640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92e00a2d30 0x7f92e00a51f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f92f00049d0 tx=0x7f92f0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 --2- 172.21.15.103:0/431686238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649d2830 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0314e0 tx=0x7f92ec032f20 comp rx=0 tx=0).stop 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 -- 172.21.15.103:0/431686238 shutdown_connections 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 --2- 172.21.15.103:0/431686238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649d2830 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 --2- 172.21.15.103:0/431686238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 --2- 172.21.15.103:0/431686238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 0x55f5649cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 -- 172.21.15.103:0/431686238 >> 172.21.15.103:0/431686238 conn(0x55f5649c6010 msgr2=0x55f5649c6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 -- 172.21.15.103:0/431686238 shutdown_connections 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 -- 172.21.15.103:0/431686238 wait complete. 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.230+0000 7f93001c4740 1 -- start start 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f93001c4740 1 Processor -- start 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f93001c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649ca9a0 0x55f5649d4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f93001c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649d50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f93001c4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f93001c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f5649d32e0 con 0x55f5649ca9a0 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f93001c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f5648b7290 con 0x55f5649cb2e0 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f93001c4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5649d3460 con 0x55f5649d0370 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f93001c4740 10 client.0 _ll_register_callbacks cb 0x55f5648e8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649ca9a0 0x55f5649d4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649ca9a0 0x55f5649d4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57590/0 (socket says 172.21.15.103:57590) 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f6ffd640 1 -- 172.21.15.103:0/2995951390 learned_addr learned my addr 172.21.15.103:0/2995951390 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649dc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:47086/0 (socket says 172.21.15.103:47086) 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f67fc640 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649d50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f6ffd640 1 -- 172.21.15.103:0/2995951390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 msgr2=0x55f5649dc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649dc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f6ffd640 1 -- 172.21.15.103:0/2995951390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 msgr2=0x55f5649d50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649d50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.231+0000 7f92f6ffd640 1 -- 172.21.15.103:0/2995951390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f564641210 con 0x55f5649ca9a0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f92f77fe640 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649ca9a0 0x55f5649d4b90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f92e80a3e50 tx=0x7f92e80973b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9820cc50420aa85c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f92daff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f92daff5640 1 -- 172.21.15.103:0/2995951390 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e809b250 con 0x55f5649ca9a0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f93001c4740 1 -- 172.21.15.103:0/2995951390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92ec031880 con 0x55f5649ca9a0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f92daff5640 1 -- 172.21.15.103:0/2995951390 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f92e8099a10 con 0x55f5649ca9a0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f92daff5640 1 -- 172.21.15.103:0/2995951390 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f92e80b4050 con 0x55f5649ca9a0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f93001c4740 10 client.15765 fetch_fsmap learned FSMap version 33 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f93001c4740 1 -- 172.21.15.103:0/2995951390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f56470d620 con 0x55f5649ca9a0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.232+0000 7f92daff5640 1 -- 172.21.15.103:0/2995951390 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec031880 con 0x55f5649ca9a0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f92daff5640 1 -- 172.21.15.103:0/2995951390 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f92e80a3ef0 con 0x55f5649ca9a0 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f93001c4740 10 client.15765 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:05:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f93001c4740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:05:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f93001c4740 20 client.15765 populate_metadata read hostname 'smithi103' 2024-09-05T02:05:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f93001c4740 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f5649e9010 0x55f5649eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f93001c4740 4 client.15765 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:05:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f93001c4740 1 -- 172.21.15.103:0/2995951390 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f56493ba20 con 0x55f5649e9010 2024-09-05T02:05:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f92f67fc640 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f5649e9010 0x55f5649eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f92f67fc640 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f5649e9010 0x55f5649eb4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b5f4bf00da4d76 server_cookie=94d0c0dc570af96e in_seq=0 out_seq=0 2024-09-05T02:05:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.233+0000 7f92daff5640 10 client.15765 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f92daff5640 1 -- 172.21.15.103:0/2995951390 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f56493ba20 con 0x55f5649e9010 2024-09-05T02:05:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f92daff5640 10 client.15765 handle_command_reply: tid=0 2024-09-05T02:05:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f93001c4740 1 client.15765 shutdown 2024-09-05T02:05:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f93001c4740 1 -- 172.21.15.103:0/2995951390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649ca9a0 msgr2=0x55f5649d4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f93001c4740 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649ca9a0 0x55f5649d4b90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f92e80a3e50 tx=0x7f92e80973b0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f93001c4740 1 -- 172.21.15.103:0/2995951390 shutdown_connections 2024-09-05T02:05:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f93001c4740 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f5649e9010 0x55f5649eb4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f93001c4740 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f93001c4740 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649d50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f93001c4740 1 --2- 172.21.15.103:0/2995951390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649ca9a0 0x55f5649d4b90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.234+0000 7f93001c4740 1 -- 172.21.15.103:0/2995951390 >> 172.21.15.103:0/2995951390 conn(0x55f5649c6010 msgr2=0x55f5649c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.235+0000 7f92daff5640 0 client.15765 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.235+0000 7f93001c4740 1 -- 172.21.15.103:0/2995951390 shutdown_connections 2024-09-05T02:05:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.235+0000 7f93001c4740 1 -- 172.21.15.103:0/2995951390 wait complete. 2024-09-05T02:05:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.235+0000 7f93001c4740 20 client.15765 trim_cache size 0 max 16384 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f93001c4740 1 Processor -- start 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f93001c4740 1 -- start start 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f93001c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649ca9a0 0x55f5649c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f93001c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649cb2e0 0x55f5649dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f93001c4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f93001c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92f81a84c0 con 0x55f5649cb2e0 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f93001c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f92f81aa180 con 0x55f5649ca9a0 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f93001c4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92f81a8b90 con 0x55f5649d0370 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f92f77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f92f77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649efd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:47090/0 (socket says 172.21.15.103:47090) 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f92f77fe640 1 -- 172.21.15.103:0/2645533415 learned_addr learned my addr 172.21.15.103:0/2645533415 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f92f67fc640 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649cb2e0 0x55f5649dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.256+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649ca9a0 0x55f5649c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92f77fe640 1 -- 172.21.15.103:0/2645533415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649ca9a0 msgr2=0x55f5649c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92f77fe640 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649ca9a0 0x55f5649c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92f77fe640 1 -- 172.21.15.103:0/2645533415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649cb2e0 msgr2=0x55f5649dc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92f77fe640 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649cb2e0 0x55f5649dc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92f77fe640 1 -- 172.21.15.103:0/2645533415 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5649c5e30 con 0x55f5649d0370 2024-09-05T02:05:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92f67fc640 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649cb2e0 0x55f5649dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92f77fe640 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649efd20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92f000fd40 tx=0x7f92f0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813a68aa3e738f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92dcff9640 1 -- 172.21.15.103:0/2645533415 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f001d910 con 0x55f5649d0370 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92dcff9640 1 -- 172.21.15.103:0/2645533415 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92f002e070 con 0x55f5649d0370 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.257+0000 7f92dcff9640 1 -- 172.21.15.103:0/2645533415 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0029de0 con 0x55f5649d0370 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 -- 172.21.15.103:0/2645533415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 msgr2=0x55f5649efd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649efd20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92f000fd40 tx=0x7f92f0017000 comp rx=0 tx=0).stop 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 -- 172.21.15.103:0/2645533415 shutdown_connections 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649d0370 0x55f5649efd20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649cb2e0 0x55f5649dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 --2- 172.21.15.103:0/2645533415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649ca9a0 0x55f5649c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 -- 172.21.15.103:0/2645533415 >> 172.21.15.103:0/2645533415 conn(0x55f5649c6010 msgr2=0x55f5649c49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 -- 172.21.15.103:0/2645533415 shutdown_connections 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 -- 172.21.15.103:0/2645533415 wait complete. 2024-09-05T02:05:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.258+0000 7f93001c4740 1 -- start start 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f93001c4740 1 Processor -- start 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f93001c4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 0x55f5649d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f93001c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649d9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f93001c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649da6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f93001c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92f81a99f0 con 0x55f5649d0370 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f93001c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f92f81a9260 con 0x55f5649cb2e0 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f93001c4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92f81a64a0 con 0x55f5649ca9a0 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 0x55f5649d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f93001c4740 10 client.0 _ll_register_callbacks cb 0x7f92e80d2970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 0x55f5649d91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:47098/0 (socket says 172.21.15.103:47098) 2024-09-05T02:05:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f6ffd640 1 -- 172.21.15.103:0/2425720860 learned_addr learned my addr 172.21.15.103:0/2425720860 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f77fe640 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649da6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f67fc640 1 -- 172.21.15.103:0/2425720860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 msgr2=0x55f5649d91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f67fc640 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 0x55f5649d91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f67fc640 1 -- 172.21.15.103:0/2425720860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 msgr2=0x55f5649da6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f67fc640 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649da6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f67fc640 1 -- 172.21.15.103:0/2425720860 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5649dacb0 con 0x55f5649cb2e0 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.259+0000 7f92f77fe640 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649da6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.260+0000 7f92f67fc640 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649d9700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02a930 tx=0x7f92ec005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20768a660b175b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.260+0000 7f92daff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.260+0000 7f92daff5640 1 -- 172.21.15.103:0/2425720860 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec035040 con 0x55f5649cb2e0 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.260+0000 7f92daff5640 1 -- 172.21.15.103:0/2425720860 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f92ec004030 con 0x55f5649cb2e0 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.260+0000 7f93001c4740 1 -- 172.21.15.103:0/2425720860 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f002e070 con 0x55f5649cb2e0 2024-09-05T02:05:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.260+0000 7f92daff5640 1 -- 172.21.15.103:0/2425720860 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f92ec061050 con 0x55f5649cb2e0 2024-09-05T02:05:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f93001c4740 10 client.25217 fetch_fsmap learned FSMap version 33 2024-09-05T02:05:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f93001c4740 1 -- 172.21.15.103:0/2425720860 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f5649dc010 con 0x55f5649cb2e0 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f92daff5640 1 -- 172.21.15.103:0/2425720860 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92f002e070 con 0x55f5649cb2e0 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f92daff5640 1 -- 172.21.15.103:0/2425720860 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f92ec045470 con 0x55f5649cb2e0 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f93001c4740 10 client.25217 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f93001c4740 10 client.25217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f93001c4740 20 client.25217 populate_metadata read hostname 'smithi103' 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f93001c4740 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f564a53ee0 0x55f564a562d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f93001c4740 4 client.25217 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f93001c4740 1 -- 172.21.15.103:0/2425720860 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f564935d70 con 0x55f564a53ee0 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.261+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f564a53ee0 0x55f564a562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.262+0000 7f92daff5640 10 client.25217 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.262+0000 7f92f6ffd640 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f564a53ee0 0x55f564a562d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5a9b161b8dc51a server_cookie=f71c19e9cf660b40 in_seq=0 out_seq=0 2024-09-05T02:05:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f92daff5640 1 -- 172.21.15.103:0/2425720860 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f564935d70 con 0x55f564a53ee0 2024-09-05T02:05:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f92daff5640 10 client.25217 handle_command_reply: tid=0 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 client.25217 shutdown 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 -- 172.21.15.103:0/2425720860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 msgr2=0x55f5649d9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649d9700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02a930 tx=0x7f92ec005850 comp rx=0 tx=0).stop 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 -- 172.21.15.103:0/2425720860 shutdown_connections 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f564a53ee0 0x55f564a562d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5649d0370 0x55f5649da6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f5649cb2e0 0x55f5649d9700 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f92daff5640 0 client.25217 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 --2- 172.21.15.103:0/2425720860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5649ca9a0 0x55f5649d91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 -- 172.21.15.103:0/2425720860 >> 172.21.15.103:0/2425720860 conn(0x55f5649c6010 msgr2=0x55f5649c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 -- 172.21.15.103:0/2425720860 shutdown_connections 2024-09-05T02:05:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 1 -- 172.21.15.103:0/2425720860 wait complete. 2024-09-05T02:05:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.351+0000 7f93001c4740 20 client.25217 trim_cache size 0 max 16384 2024-09-05T02:05:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:05:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:05:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:05:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:05:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 -- 172.21.15.103:0/1365482059 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92e00a2d30 msgr2=0x7f92e00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92e00a2d30 0x7f92e00a51f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f92f00049d0 tx=0x7f92f0009500 comp rx=0 tx=0).stop 2024-09-05T02:05:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 -- 172.21.15.103:0/1365482059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f8100a50 msgr2=0x7f92f810b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f8100a50 0x7f92f810b3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f92e800e9c0 tx=0x7f92e800ee80 comp rx=0 tx=0).stop 2024-09-05T02:05:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 -- 172.21.15.103:0/1365482059 shutdown_connections 2024-09-05T02:05:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92f8102600 0x7f92f8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92f8101c40 0x7f92f8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f92e00a2d30 0x7f92e00a51f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 --2- 172.21.15.103:0/1365482059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f8100a50 0x7f92f810b3d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 -- 172.21.15.103:0/1365482059 >> 172.21.15.103:0/1365482059 conn(0x7f92f80fc1b0 msgr2=0x7f92f8105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 -- 172.21.15.103:0/1365482059 shutdown_connections 2024-09-05T02:05:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:36.352+0000 7f92fd427640 1 -- 172.21.15.103:0/1365482059 wait complete. 2024-09-05T02:05:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:36 smithi167 ceph-mon[35122]: pgmap v445: 129 pgs: 129 active+clean; 157 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 349 op/s 2024-09-05T02:05:36.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:36 smithi139 ceph-mon[34262]: pgmap v445: 129 pgs: 129 active+clean; 157 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 349 op/s 2024-09-05T02:05:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:36 smithi103 ceph-mon[30902]: pgmap v445: 129 pgs: 129 active+clean; 157 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 349 op/s 2024-09-05T02:05:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:38 smithi167 ceph-mon[35122]: pgmap v446: 129 pgs: 129 active+clean; 159 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2024-09-05T02:05:38.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:38 smithi139 ceph-mon[34262]: pgmap v446: 129 pgs: 129 active+clean; 159 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2024-09-05T02:05:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:38 smithi103 ceph-mon[30902]: pgmap v446: 129 pgs: 129 active+clean; 159 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2024-09-05T02:05:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:40 smithi167 ceph-mon[35122]: pgmap v447: 129 pgs: 129 active+clean; 166 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 330 op/s 2024-09-05T02:05:40.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:40 smithi139 ceph-mon[34262]: pgmap v447: 129 pgs: 129 active+clean; 166 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 330 op/s 2024-09-05T02:05:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:40 smithi103 ceph-mon[30902]: pgmap v447: 129 pgs: 129 active+clean; 166 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 330 op/s 2024-09-05T02:05:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:42 smithi167 ceph-mon[35122]: pgmap v448: 129 pgs: 129 active+clean; 172 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 345 op/s 2024-09-05T02:05:42.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:42 smithi139 ceph-mon[34262]: pgmap v448: 129 pgs: 129 active+clean; 172 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 345 op/s 2024-09-05T02:05:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:42 smithi103 ceph-mon[30902]: pgmap v448: 129 pgs: 129 active+clean; 172 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 345 op/s 2024-09-05T02:05:43.874 INFO:tasks.workunit.client.0.smithi103.stdout: 42 12094 348 6464 242 3814 233 2024-09-05T02:05:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:44 smithi167 ceph-mon[35122]: pgmap v449: 129 pgs: 129 active+clean; 174 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-05T02:05:44.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:44 smithi139 ceph-mon[34262]: pgmap v449: 129 pgs: 129 active+clean; 174 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-05T02:05:44.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:44 smithi103 ceph-mon[30902]: pgmap v449: 129 pgs: 129 active+clean; 174 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-05T02:05:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:46 smithi167 ceph-mon[35122]: pgmap v450: 129 pgs: 129 active+clean; 184 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 329 op/s 2024-09-05T02:05:46.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:46 smithi139 ceph-mon[34262]: pgmap v450: 129 pgs: 129 active+clean; 184 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 329 op/s 2024-09-05T02:05:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:46 smithi103 ceph-mon[30902]: pgmap v450: 129 pgs: 129 active+clean; 184 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 329 op/s 2024-09-05T02:05:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:48 smithi167 ceph-mon[35122]: pgmap v451: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-09-05T02:05:48.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:48 smithi139 ceph-mon[34262]: pgmap v451: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-09-05T02:05:48.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:48 smithi103 ceph-mon[30902]: pgmap v451: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-09-05T02:05:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.482+0000 7f5506da6640 1 -- 172.21.15.103:0/2121005523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55001025f0 msgr2=0x7f5500108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.482+0000 7f5506da6640 1 --2- 172.21.15.103:0/2121005523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55001025f0 0x7f5500108e80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009ca0 tx=0x7f54f802a8c0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.482+0000 7f5506da6640 1 -- 172.21.15.103:0/2121005523 shutdown_connections 2024-09-05T02:05:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.482+0000 7f5506da6640 1 --2- 172.21.15.103:0/2121005523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55001025f0 0x7f5500108e80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.482+0000 7f5506da6640 1 --2- 172.21.15.103:0/2121005523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5500101c30 0x7f55001020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.482+0000 7f5506da6640 1 --2- 172.21.15.103:0/2121005523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5500100a40 0x7f5500100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.482+0000 7f5506da6640 1 -- 172.21.15.103:0/2121005523 >> 172.21.15.103:0/2121005523 conn(0x7f55000fc1e0 msgr2=0x7f55000fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.482+0000 7f5506da6640 1 -- 172.21.15.103:0/2121005523 shutdown_connections 2024-09-05T02:05:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.482+0000 7f5506da6640 1 -- 172.21.15.103:0/2121005523 wait complete. 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.483+0000 7f5506da6640 1 Processor -- start 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.483+0000 7f5506da6640 1 -- start start 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.483+0000 7f5506da6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5500100a40 0x7f5500195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.483+0000 7f5506da6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5500101c30 0x7f5500195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.483+0000 7f5506da6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55001025f0 0x7f550019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.483+0000 7f5506da6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f550010b4b0 con 0x7f5500101c30 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.483+0000 7f5506da6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f550010b330 con 0x7f5500100a40 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.483+0000 7f5506da6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f550010b630 con 0x7f55001025f0 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f54fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5500101c30 0x7f5500195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f5504b42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5500100a40 0x7f5500195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f54fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5500101c30 0x7f5500195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60918/0 (socket says 172.21.15.103:60918) 2024-09-05T02:05:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f54fffff640 1 -- 172.21.15.103:0/107863863 learned_addr learned my addr 172.21.15.103:0/107863863 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f5505343640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55001025f0 0x7f550019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f5504b42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5500100a40 0x7f5500195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:49820/0 (socket says 172.21.15.103:49820) 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f5505343640 1 -- 172.21.15.103:0/107863863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5500100a40 msgr2=0x7f5500195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f5505343640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5500100a40 0x7f5500195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f5505343640 1 -- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5500101c30 msgr2=0x7f5500195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f5505343640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5500101c30 0x7f5500195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f5505343640 1 -- 172.21.15.103:0/107863863 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55001a1ac0 con 0x7f55001025f0 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f54fffff640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5500101c30 0x7f5500195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f5505343640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55001025f0 0x7f550019d040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54f8031970 tx=0x7f54f8009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.484+0000 7f54fdffb640 1 -- 172.21.15.103:0/107863863 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8003c30 con 0x7f55001025f0 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.485+0000 7f54fdffb640 1 -- 172.21.15.103:0/107863863 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f8003dd0 con 0x7f55001025f0 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.485+0000 7f5506da6640 1 -- 172.21.15.103:0/107863863 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55001a1d50 con 0x7f55001025f0 2024-09-05T02:05:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.485+0000 7f54fdffb640 1 -- 172.21.15.103:0/107863863 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8030b70 con 0x7f55001025f0 2024-09-05T02:05:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.485+0000 7f5506da6640 1 -- 172.21.15.103:0/107863863 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55001a2260 con 0x7f55001025f0 2024-09-05T02:05:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.488+0000 7f5509c1f740 1 Processor -- start 2024-09-05T02:05:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.488+0000 7f54fdffb640 1 -- 172.21.15.103:0/107863863 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f54f8030d10 con 0x7f55001025f0 2024-09-05T02:05:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.488+0000 7f5509c1f740 1 -- start start 2024-09-05T02:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.489+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 0x55bcacb775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.489+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb7c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.489+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb7cb80 0x55bcacb7f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.489+0000 7f5509c1f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bcacad22f0 con 0x55bcacb77af0 2024-09-05T02:05:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.489+0000 7f5509c1f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bcaca7a270 con 0x55bcacb771b0 2024-09-05T02:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.489+0000 7f5509c1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bcacace2b0 con 0x55bcacb7cb80 2024-09-05T02:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.489+0000 7f54fffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb7cb80 0x55bcacb7f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.489+0000 7f5504b42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb7c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.490+0000 7f5504b42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb7c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60924/0 (socket says 172.21.15.103:60924) 2024-09-05T02:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.490+0000 7f5504b42640 1 -- 172.21.15.103:0/2540678348 learned_addr learned my addr 172.21.15.103:0/2540678348 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.490+0000 7f54fffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb7cb80 0x55bcacb7f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56628/0 (socket says 172.21.15.103:56628) 2024-09-05T02:05:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.490+0000 7f54fffff640 1 --2- 172.21.15.103:0/2540678348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 0x55bcacb775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.490+0000 7f54fffff640 1 -- 172.21.15.103:0/2540678348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 msgr2=0x55bcacb775b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.490+0000 7f54fffff640 1 --2- 172.21.15.103:0/2540678348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 0x55bcacb775b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.490+0000 7f54fffff640 1 -- 172.21.15.103:0/2540678348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 msgr2=0x55bcacb7c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.490+0000 7f54fffff640 1 --2- 172.21.15.103:0/2540678348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb7c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.490+0000 7f54fffff640 1 -- 172.21.15.103:0/2540678348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcaca79340 con 0x55bcacb7cb80 2024-09-05T02:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.491+0000 7f5504b42640 1 --2- 172.21.15.103:0/2540678348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb7c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.491+0000 7f54fffff640 1 --2- 172.21.15.103:0/2540678348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb7cb80 0x55bcacb7f040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f54ec013c40 tx=0x7f54ec005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88c4870e19ed8be1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.491+0000 7f54fdffb640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f54f00a2d30 0x7f54f00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.491+0000 7f54eaffd640 1 -- 172.21.15.103:0/2540678348 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec013df0 con 0x55bcacb7cb80 2024-09-05T02:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.491+0000 7f54eaffd640 1 -- 172.21.15.103:0/2540678348 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54ec0054d0 con 0x55bcacb7cb80 2024-09-05T02:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.491+0000 7f54eaffd640 1 -- 172.21.15.103:0/2540678348 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec004a00 con 0x55bcacb7cb80 2024-09-05T02:05:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.491+0000 7f5504b42640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f54f00a2d30 0x7f54f00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5509c1f740 1 -- 172.21.15.103:0/2540678348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb7cb80 msgr2=0x55bcacb7f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2540678348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb7cb80 0x55bcacb7f040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f54ec013c40 tx=0x7f54ec005f90 comp rx=0 tx=0).stop 2024-09-05T02:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5509c1f740 1 -- 172.21.15.103:0/2540678348 shutdown_connections 2024-09-05T02:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2540678348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb7cb80 0x55bcacb7f040 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f54fdffb640 1 -- 172.21.15.103:0/107863863 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f54f80f0740 con 0x7f55001025f0 2024-09-05T02:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2540678348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb7c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2540678348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 0x55bcacb775b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5504b42640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f54f00a2d30 0x7f54f00a51f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55bcacb78520 tx=0x7f54f400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5509c1f740 1 -- 172.21.15.103:0/2540678348 >> 172.21.15.103:0/2540678348 conn(0x55bcacb6a010 msgr2=0x55bcacb6a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5509c1f740 1 -- 172.21.15.103:0/2540678348 shutdown_connections 2024-09-05T02:05:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.492+0000 7f5509c1f740 1 -- 172.21.15.103:0/2540678348 wait complete. 2024-09-05T02:05:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.493+0000 7f5509c1f740 1 -- start start 2024-09-05T02:05:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.493+0000 7f5509c1f740 1 Processor -- start 2024-09-05T02:05:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb771b0 0x55bcacb81400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb81940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb7cb80 0x55bcacb88980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5509c1f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bcacb7fb50 con 0x55bcacb7cb80 2024-09-05T02:05:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5509c1f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bcaca5b290 con 0x55bcacb77af0 2024-09-05T02:05:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5509c1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bcacb7fcd0 con 0x55bcacb771b0 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5509c1f740 10 client.0 _ll_register_callbacks cb 0x55bcaca8c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5505343640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb7cb80 0x55bcacb88980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f54fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb81940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5505343640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb7cb80 0x55bcacb88980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60938/0 (socket says 172.21.15.103:60938) 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5505343640 1 -- 172.21.15.103:0/2530662450 learned_addr learned my addr 172.21.15.103:0/2530662450 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f54fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb81940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:49838/0 (socket says 172.21.15.103:49838) 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5504b42640 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb771b0 0x55bcacb81400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f54fffff640 1 -- 172.21.15.103:0/2530662450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb771b0 msgr2=0x55bcacb81400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f54fffff640 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb771b0 0x55bcacb81400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f54fffff640 1 -- 172.21.15.103:0/2530662450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb7cb80 msgr2=0x55bcacb88980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f54fffff640 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb7cb80 0x55bcacb88980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f54fffff640 1 -- 172.21.15.103:0/2530662450 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcac7e5210 con 0x55bcacb77af0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.494+0000 7f5505343640 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb7cb80 0x55bcacb88980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f54fffff640 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb81940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54ec01d000 tx=0x7f54ec005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec4dc39219a7cee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f54dffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f54dffff640 1 -- 172.21.15.103:0/2530662450 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec020460 con 0x55bcacb77af0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f5509c1f740 1 -- 172.21.15.103:0/2530662450 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ec0054d0 con 0x55bcacb77af0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f54dffff640 1 -- 172.21.15.103:0/2530662450 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f54ec020600 con 0x55bcacb77af0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f54dffff640 1 -- 172.21.15.103:0/2530662450 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54ec030050 con 0x55bcacb77af0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f5509c1f740 10 client.25229 fetch_fsmap learned FSMap version 33 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f5509c1f740 1 -- 172.21.15.103:0/2530662450 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bcac8b1620 con 0x55bcacb77af0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f54dffff640 1 -- 172.21.15.103:0/2530662450 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54ec0054d0 con 0x55bcacb77af0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.495+0000 7f54dffff640 1 -- 172.21.15.103:0/2530662450 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f54ec0108f0 con 0x55bcacb77af0 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.496+0000 7f5509c1f740 10 client.25229 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:05:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.496+0000 7f5509c1f740 10 client.25229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.496+0000 7f5509c1f740 20 client.25229 populate_metadata read hostname 'smithi103' 2024-09-05T02:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.496+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55bcacb8ce70 0x55bcacb8f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.496+0000 7f5509c1f740 4 client.25229 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.496+0000 7f5509c1f740 1 -- 172.21.15.103:0/2530662450 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcacadfa20 con 0x55bcacb8ce70 2024-09-05T02:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.496+0000 7f5504b42640 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55bcacb8ce70 0x55bcacb8f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.496+0000 7f54dffff640 10 client.25229 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.496+0000 7f5504b42640 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55bcacb8ce70 0x55bcacb8f260 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43bf6a2f88ac4c52 server_cookie=2d2b115518523333 in_seq=0 out_seq=0 2024-09-05T02:05:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f54dffff640 1 -- 172.21.15.103:0/2530662450 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcacadfa20 con 0x55bcacb8ce70 2024-09-05T02:05:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f54dffff640 10 client.25229 handle_command_reply: tid=0 2024-09-05T02:05:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 client.25229 shutdown 2024-09-05T02:05:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 -- 172.21.15.103:0/2530662450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 msgr2=0x55bcacb81940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb81940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54ec01d000 tx=0x7f54ec005780 comp rx=0 tx=0).stop 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 -- 172.21.15.103:0/2530662450 shutdown_connections 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55bcacb8ce70 0x55bcacb8f260 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb7cb80 0x55bcacb88980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f54dffff640 0 client.25229 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb81940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 --2- 172.21.15.103:0/2530662450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb771b0 0x55bcacb81400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 -- 172.21.15.103:0/2530662450 >> 172.21.15.103:0/2530662450 conn(0x55bcacb6a010 msgr2=0x55bcacb6b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 -- 172.21.15.103:0/2530662450 shutdown_connections 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.497+0000 7f5509c1f740 1 -- 172.21.15.103:0/2530662450 wait complete. 2024-09-05T02:05:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.498+0000 7f5509c1f740 20 client.25229 trim_cache size 0 max 16384 2024-09-05T02:05:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.522+0000 7f5509c1f740 1 Processor -- start 2024-09-05T02:05:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.522+0000 7f5509c1f740 1 -- start start 2024-09-05T02:05:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb6aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 0x55bcacb88940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb6b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5509c1f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55001a83d0 con 0x55bcacb77af0 2024-09-05T02:05:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5504b42640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb6aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f54fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 0x55bcacb88940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5504b42640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb6aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56640/0 (socket says 172.21.15.103:56640) 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f54fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 0x55bcacb88940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:49844/0 (socket says 172.21.15.103:49844) 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5504b42640 1 -- 172.21.15.103:0/857595181 learned_addr learned my addr 172.21.15.103:0/857595181 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5509c1f740 1 -- 172.21.15.103:0/857595181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f55001aa090 con 0x55bcacb771b0 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5505343640 1 --2- 172.21.15.103:0/857595181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5509c1f740 1 -- 172.21.15.103:0/857595181 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55001a8aa0 con 0x55bcacb6a010 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5504b42640 1 -- 172.21.15.103:0/857595181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 msgr2=0x55bcacb88940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5504b42640 1 --2- 172.21.15.103:0/857595181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 0x55bcacb88940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.523+0000 7f5504b42640 1 -- 172.21.15.103:0/857595181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 msgr2=0x55bcacb6b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.524+0000 7f5504b42640 1 --2- 172.21.15.103:0/857595181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb6b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.524+0000 7f5504b42640 1 -- 172.21.15.103:0/857595181 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcacb8f8e0 con 0x55bcacb6a010 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.524+0000 7f5504b42640 1 --2- 172.21.15.103:0/857595181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb6aec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f54f4019040 tx=0x7f54f4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3f79da69b427b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.524+0000 7f54ea7fc640 1 -- 172.21.15.103:0/857595181 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4031070 con 0x55bcacb6a010 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.524+0000 7f54ea7fc640 1 -- 172.21.15.103:0/857595181 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f401d040 con 0x55bcacb6a010 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.524+0000 7f54ea7fc640 1 -- 172.21.15.103:0/857595181 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f402c270 con 0x55bcacb6a010 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.525+0000 7f5509c1f740 1 -- 172.21.15.103:0/857595181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 msgr2=0x55bcacb6aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.525+0000 7f5509c1f740 1 --2- 172.21.15.103:0/857595181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb6aec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f54f4019040 tx=0x7f54f4017000 comp rx=0 tx=0).stop 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.525+0000 7f5509c1f740 1 -- 172.21.15.103:0/857595181 shutdown_connections 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.525+0000 7f5509c1f740 1 --2- 172.21.15.103:0/857595181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb77af0 0x55bcacb6b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.525+0000 7f5509c1f740 1 --2- 172.21.15.103:0/857595181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb771b0 0x55bcacb88940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.525+0000 7f5509c1f740 1 --2- 172.21.15.103:0/857595181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb6aec0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.525+0000 7f5509c1f740 1 -- 172.21.15.103:0/857595181 >> 172.21.15.103:0/857595181 conn(0x55bcacb7c6d0 msgr2=0x55bcacb78cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.525+0000 7f5509c1f740 1 -- 172.21.15.103:0/857595181 shutdown_connections 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.525+0000 7f5509c1f740 1 -- 172.21.15.103:0/857595181 wait complete. 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.526+0000 7f5509c1f740 1 -- start start 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.526+0000 7f5509c1f740 1 Processor -- start 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.526+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb85fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.526+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb771b0 0x55bcacb86500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.526+0000 7f5509c1f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb87510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.526+0000 7f5509c1f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55001a9900 con 0x55bcacb771b0 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.526+0000 7f5509c1f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f55001a9170 con 0x55bcacb77af0 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.526+0000 7f5509c1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55001a63b0 con 0x55bcacb6a010 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.526+0000 7f5509c1f740 10 client.0 _ll_register_callbacks cb 0x7f54f400b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:05:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb771b0 0x55bcacb86500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb771b0 0x55bcacb86500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60956/0 (socket says 172.21.15.103:60956) 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54fffff640 1 -- 172.21.15.103:0/3213407815 learned_addr learned my addr 172.21.15.103:0/3213407815 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f5505343640 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb87510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f5504b42640 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb85fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54fffff640 1 -- 172.21.15.103:0/3213407815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 msgr2=0x55bcacb85fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54fffff640 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb85fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54fffff640 1 -- 172.21.15.103:0/3213407815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 msgr2=0x55bcacb87510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54fffff640 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb87510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54fffff640 1 -- 172.21.15.103:0/3213407815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcacb87b20 con 0x55bcacb771b0 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f5504b42640 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb85fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f5505343640 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb87510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54fffff640 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb771b0 0x55bcacb86500 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000c00 tx=0x7f54ec00fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcf60bb6875bd21 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54dffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54dffff640 1 -- 172.21.15.103:0/3213407815 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec02d020 con 0x55bcacb771b0 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f54dffff640 1 -- 172.21.15.103:0/3213407815 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f54ec0050c0 con 0x55bcacb771b0 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.527+0000 7f5509c1f740 1 -- 172.21.15.103:0/3213407815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f401d040 con 0x55bcacb771b0 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.528+0000 7f54dffff640 1 -- 172.21.15.103:0/3213407815 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54ec0136f0 con 0x55bcacb771b0 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.528+0000 7f5509c1f740 10 client.15789 fetch_fsmap learned FSMap version 33 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.528+0000 7f5509c1f740 1 -- 172.21.15.103:0/3213407815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bcacbf7b40 con 0x55bcacb771b0 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.528+0000 7f54dffff640 1 -- 172.21.15.103:0/3213407815 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f401d040 con 0x55bcacb771b0 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.528+0000 7f54dffff640 1 -- 172.21.15.103:0/3213407815 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f54ec00e6b0 con 0x55bcacb771b0 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.529+0000 7f5509c1f740 10 client.15789 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.529+0000 7f5509c1f740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.529+0000 7f5509c1f740 20 client.15789 populate_metadata read hostname 'smithi103' 2024-09-05T02:05:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.529+0000 7f5509c1f740 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55bcacb7cb80 0x55bcacbfa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.529+0000 7f5509c1f740 4 client.15789 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:05:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.529+0000 7f5509c1f740 1 -- 172.21.15.103:0/3213407815 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcacad9d70 con 0x55bcacb7cb80 2024-09-05T02:05:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.529+0000 7f5504b42640 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55bcacb7cb80 0x55bcacbfa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.529+0000 7f5504b42640 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55bcacb7cb80 0x55bcacbfa4e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d470adae9175ea57 server_cookie=ffc2977001c23081 in_seq=0 out_seq=0 2024-09-05T02:05:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.529+0000 7f54dffff640 10 client.15789 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.628+0000 7f54dffff640 1 -- 172.21.15.103:0/3213407815 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcacad9d70 con 0x55bcacb7cb80 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.628+0000 7f54dffff640 10 client.15789 handle_command_reply: tid=0 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.628+0000 7f5509c1f740 1 client.15789 shutdown 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.628+0000 7f5509c1f740 1 -- 172.21.15.103:0/3213407815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb771b0 msgr2=0x55bcacb86500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.628+0000 7f5509c1f740 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb771b0 0x55bcacb86500 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000c00 tx=0x7f54ec00fe50 comp rx=0 tx=0).stop 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f5509c1f740 1 -- 172.21.15.103:0/3213407815 shutdown_connections 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f5509c1f740 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55bcacb7cb80 0x55bcacbfa4e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f5509c1f740 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcacb77af0 0x55bcacb87510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f5509c1f740 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcacb771b0 0x55bcacb86500 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f54dffff640 0 client.15789 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f5509c1f740 1 --2- 172.21.15.103:0/3213407815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bcacb6a010 0x55bcacb85fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f5509c1f740 1 -- 172.21.15.103:0/3213407815 >> 172.21.15.103:0/3213407815 conn(0x55bcacb7c6d0 msgr2=0x55bcacb689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f5509c1f740 1 -- 172.21.15.103:0/3213407815 shutdown_connections 2024-09-05T02:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f5509c1f740 1 -- 172.21.15.103:0/3213407815 wait complete. 2024-09-05T02:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.629+0000 7f5509c1f740 20 client.15789 trim_cache size 0 max 16384 2024-09-05T02:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 -- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f54f00a2d30 msgr2=0x7f54f00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f54f00a2d30 0x7f54f00a51f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55bcacb78520 tx=0x7f54f400d040 comp rx=0 tx=0).stop 2024-09-05T02:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 -- 172.21.15.103:0/107863863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55001025f0 msgr2=0x7f550019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55001025f0 0x7f550019d040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54f8031970 tx=0x7f54f8009800 comp rx=0 tx=0).stop 2024-09-05T02:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 -- 172.21.15.103:0/107863863 shutdown_connections 2024-09-05T02:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f54f00a2d30 0x7f54f00a51f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55001025f0 0x7f550019d040 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5500101c30 0x7f5500195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 --2- 172.21.15.103:0/107863863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5500100a40 0x7f5500195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 -- 172.21.15.103:0/107863863 >> 172.21.15.103:0/107863863 conn(0x7f55000fc1e0 msgr2=0x7f5500105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 -- 172.21.15.103:0/107863863 shutdown_connections 2024-09-05T02:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:05:49.630+0000 7f5506da6640 1 -- 172.21.15.103:0/107863863 wait complete. 2024-09-05T02:05:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:50 smithi167 ceph-mon[35122]: pgmap v452: 129 pgs: 129 active+clean; 190 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 326 op/s 2024-09-05T02:05:50.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:50 smithi139 ceph-mon[34262]: pgmap v452: 129 pgs: 129 active+clean; 190 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 326 op/s 2024-09-05T02:05:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:50 smithi103 ceph-mon[30902]: pgmap v452: 129 pgs: 129 active+clean; 190 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 326 op/s 2024-09-05T02:05:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:52 smithi167 ceph-mon[35122]: pgmap v453: 129 pgs: 129 active+clean; 193 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-09-05T02:05:52.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:52 smithi139 ceph-mon[34262]: pgmap v453: 129 pgs: 129 active+clean; 193 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-09-05T02:05:52.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:52 smithi103 ceph-mon[30902]: pgmap v453: 129 pgs: 129 active+clean; 193 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-09-05T02:05:53.874 INFO:tasks.workunit.client.0.smithi103.stdout: 44 12491 88 7327 130 3634 221 2024-09-05T02:05:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:54 smithi167 ceph-mon[35122]: pgmap v454: 129 pgs: 129 active+clean; 197 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2024-09-05T02:05:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:54 smithi139 ceph-mon[34262]: pgmap v454: 129 pgs: 129 active+clean; 197 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2024-09-05T02:05:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:54 smithi103 ceph-mon[30902]: pgmap v454: 129 pgs: 129 active+clean; 197 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2024-09-05T02:05:55.664 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T02:05:55.664 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.75 2024-09-05T02:05:55.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:05:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:05:56.413 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:56 smithi103 ceph-mon[30902]: pgmap v455: 129 pgs: 129 active+clean; 203 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 362 op/s 2024-09-05T02:05:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:56 smithi167 ceph-mon[35122]: pgmap v455: 129 pgs: 129 active+clean; 203 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 362 op/s 2024-09-05T02:05:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:56 smithi139 ceph-mon[34262]: pgmap v455: 129 pgs: 129 active+clean; 203 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 362 op/s 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/1837650656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db4101df0 msgr2=0x7f1db41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/1837650656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db4101df0 0x7f1db41021d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009900 tx=0x7f1db002a3f0 comp rx=0 tx=0).stop 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/1837650656 shutdown_connections 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/1837650656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1db4106f20 0x7f1db410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/1837650656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db41027a0 0x7f1db41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/1837650656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db4101df0 0x7f1db41021d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/1837650656 >> 172.21.15.103:0/1837650656 conn(0x7f1db40fd8b0 msgr2=0x7f1db40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/1837650656 shutdown_connections 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/1837650656 wait complete. 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.806+0000 7f1dbbbe4640 1 Processor -- start 2024-09-05T02:05:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1dbbbe4640 1 -- start start 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1dbbbe4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db4101df0 0x7f1db4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1dbbbe4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1db41027a0 0x7f1db4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1dbbbe4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db4106f20 0x7f1db419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1dbbbe4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4111e80 con 0x7f1db4106f20 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1dbbbe4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4111d00 con 0x7f1db4101df0 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1dbbbe4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4112000 con 0x7f1db41027a0 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1db917f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1db41027a0 0x7f1db4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1db917f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1db41027a0 0x7f1db4198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56672/0 (socket says 172.21.15.103:56672) 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1db917f640 1 -- 172.21.15.103:0/2865327199 learned_addr learned my addr 172.21.15.103:0/2865327199 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.807+0000 7f1db9980640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db4101df0 0x7f1db4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.808+0000 7f1dba181640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db4106f20 0x7f1db419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.808+0000 7f1db917f640 1 -- 172.21.15.103:0/2865327199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db4101df0 msgr2=0x7f1db4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.808+0000 7f1db917f640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db4101df0 0x7f1db4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.808+0000 7f1db917f640 1 -- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db4106f20 msgr2=0x7f1db419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.808+0000 7f1db917f640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db4106f20 0x7f1db419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.808+0000 7f1db917f640 1 -- 172.21.15.103:0/2865327199 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db419d2d0 con 0x7f1db41027a0 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.808+0000 7f1db9980640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db4101df0 0x7f1db4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.808+0000 7f1dba181640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db4106f20 0x7f1db419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.808+0000 7f1db917f640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1db41027a0 0x7f1db4198860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1da400ea60 tx=0x7f1da400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.809+0000 7f1daaffd640 1 -- 172.21.15.103:0/2865327199 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da400cde0 con 0x7f1db41027a0 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.809+0000 7f1daaffd640 1 -- 172.21.15.103:0/2865327199 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1da40035f0 con 0x7f1db41027a0 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.809+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db419d5c0 con 0x7f1db41027a0 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.809+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db41a4ea0 con 0x7f1db41027a0 2024-09-05T02:05:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.809+0000 7f1daaffd640 1 -- 172.21.15.103:0/2865327199 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4010690 con 0x7f1db41027a0 2024-09-05T02:05:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.810+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db41021d0 con 0x7f1db41027a0 2024-09-05T02:05:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.812+0000 7f1daaffd640 1 -- 172.21.15.103:0/2865327199 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1da40037a0 con 0x7f1db41027a0 2024-09-05T02:05:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.812+0000 7f1daaffd640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1da00a2c00 0x7f1da00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:05:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.813+0000 7f1daaffd640 1 -- 172.21.15.103:0/2865327199 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1da40cf6e0 con 0x7f1db41027a0 2024-09-05T02:05:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.813+0000 7f1db9980640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1da00a2c00 0x7f1da00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:05:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.813+0000 7f1db9980640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1da00a2c00 0x7f1da00a50c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1db0005ab0 tx=0x7f1db0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:05:56.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:56.818+0000 7f1daaffd640 1 -- 172.21.15.103:0/2865327199 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1da409c290 con 0x7f1db41027a0 2024-09-05T02:05:57.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.159+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1db402d7c0 con 0x7f1db41027a0 2024-09-05T02:05:57.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.160+0000 7f1daaffd640 1 -- 172.21.15.103:0/2865327199 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12012 (secure 0 0 0) 0x7f1da409bb90 con 0x7f1db41027a0 2024-09-05T02:05:57.161 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:05:57.161 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":33,"btime":"2024-09-05T02:04:34:292690+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:04:33.568912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:05:57.161 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 33 2024-09-05T02:05:57.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1da00a2c00 msgr2=0x7f1da00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1da00a2c00 0x7f1da00a50c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1db0005ab0 tx=0x7f1db0005a20 comp rx=0 tx=0).stop 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1db41027a0 msgr2=0x7f1db4198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1db41027a0 0x7f1db4198860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1da400ea60 tx=0x7f1da400ef20 comp rx=0 tx=0).stop 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 shutdown_connections 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db4106f20 0x7f1db419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1da00a2c00 0x7f1da00a50c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1db41027a0 0x7f1db4198860 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 --2- 172.21.15.103:0/2865327199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db4101df0 0x7f1db4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 >> 172.21.15.103:0/2865327199 conn(0x7f1db40fd8b0 msgr2=0x7f1db40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 shutdown_connections 2024-09-05T02:05:57.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:05:57.163+0000 7f1dbbbe4640 1 -- 172.21.15.103:0/2865327199 wait complete. 2024-09-05T02:05:57.511 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:57 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2865327199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:05:57.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:57 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2865327199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:05:57.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:57 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2865327199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:05:58.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:05:58 smithi103 ceph-mon[30902]: pgmap v456: 129 pgs: 129 active+clean; 203 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-09-05T02:05:58.512 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0' 2024-09-05T02:05:58.512 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0 2024-09-05T02:05:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:05:58 smithi167 ceph-mon[35122]: pgmap v456: 129 pgs: 129 active+clean; 203 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-09-05T02:05:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:05:58 smithi139 ceph-mon[34262]: pgmap v456: 129 pgs: 129 active+clean; 203 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-09-05T02:05:58.870 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:06:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:00 smithi167 ceph-mon[35122]: pgmap v457: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-09-05T02:06:00.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:00 smithi139 ceph-mon[34262]: pgmap v457: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-09-05T02:06:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:00 smithi103 ceph-mon[30902]: pgmap v457: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-09-05T02:06:01.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.330+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3094583576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654106a80 msgr2=0x7ff654106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.330+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3094583576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654106a80 0x7ff654106e60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff650009930 tx=0x7ff65002e830 comp rx=0 tx=0).stop 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.331+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3094583576 shutdown_connections 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.331+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3094583576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff654101420 0x7ff65410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.331+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3094583576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff654100a80 0x7ff654100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.331+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3094583576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654106a80 0x7ff654106e60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.331+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3094583576 >> 172.21.15.103:0/3094583576 conn(0x7ff6540fc7a0 msgr2=0x7ff6540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.331+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3094583576 shutdown_connections 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.331+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3094583576 wait complete. 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.331+0000 7ff65c4a3640 1 Processor -- start 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.331+0000 7ff65c4a3640 1 -- start start 2024-09-05T02:06:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65c4a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654100a80 0x7ff6541961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65c4a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff654101420 0x7ff654196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65c4a3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff654106a80 0x7ff65419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65c4a3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff65410fc80 con 0x7ff654100a80 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65c4a3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff65410fb00 con 0x7ff654101420 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65c4a3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff65410fe00 con 0x7ff654106a80 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65a23f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654100a80 0x7ff6541961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65a23f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654100a80 0x7ff6541961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43598/0 (socket says 172.21.15.103:43598) 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65a23f640 1 -- 172.21.15.103:0/3680144533 learned_addr learned my addr 172.21.15.103:0/3680144533 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff659a3e640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff654101420 0x7ff654196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65aa40640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff654106a80 0x7ff65419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65a23f640 1 -- 172.21.15.103:0/3680144533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff654106a80 msgr2=0x7ff65419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65a23f640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff654106a80 0x7ff65419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65a23f640 1 -- 172.21.15.103:0/3680144533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff654101420 msgr2=0x7ff654196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65a23f640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff654101420 0x7ff654196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65a23f640 1 -- 172.21.15.103:0/3680144533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65419b170 con 0x7ff654100a80 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65aa40640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff654106a80 0x7ff65419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.332+0000 7ff65a23f640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654100a80 0x7ff6541961c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff650005ab0 tx=0x7ff650004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.333+0000 7ff64b7fe640 1 -- 172.21.15.103:0/3680144533 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650047020 con 0x7ff654100a80 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.333+0000 7ff64b7fe640 1 -- 172.21.15.103:0/3680144533 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6500044d0 con 0x7ff654100a80 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.333+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3680144533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65419b400 con 0x7ff654100a80 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.333+0000 7ff64b7fe640 1 -- 172.21.15.103:0/3680144533 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650003850 con 0x7ff654100a80 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.333+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3680144533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6541a2d40 con 0x7ff654100a80 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.335+0000 7ff64b7fe640 1 -- 172.21.15.103:0/3680144533 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff65004a020 con 0x7ff654100a80 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.336+0000 7ff65e87c740 1 Processor -- start 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.336+0000 7ff65e87c740 1 -- start start 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.336+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a307fac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.336+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a308054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65aa40640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a307fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65aa40640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a307fac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43602/0 (socket says 172.21.15.103:43602) 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30807e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65e87c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a3075f160 con 0x560a307fa850 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65e87c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a30706720 con 0x560a307fb170 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65e87c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560a3075b120 con 0x560a30805a30 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65aa40640 1 -- 172.21.15.103:0/540410063 learned_addr learned my addr 172.21.15.103:0/540410063 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65a23f640 1 --2- 172.21.15.103:0/540410063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a308054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff659a3e640 1 --2- 172.21.15.103:0/540410063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30807e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff64b7fe640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff6400a2c00 0x7ff6400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65aa40640 1 -- 172.21.15.103:0/540410063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 msgr2=0x560a30807e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65aa40640 1 --2- 172.21.15.103:0/540410063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30807e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65aa40640 1 -- 172.21.15.103:0/540410063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 msgr2=0x560a308054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65aa40640 1 --2- 172.21.15.103:0/540410063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a308054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65aa40640 1 -- 172.21.15.103:0/540410063 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a304d0ba0 con 0x560a307fa850 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff64b7fe640 1 -- 172.21.15.103:0/3680144533 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff6500f3d60 con 0x7ff654100a80 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.337+0000 7ff65aa40640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff6400a2c00 0x7ff6400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.338+0000 7ff65aa40640 1 --2- 172.21.15.103:0/540410063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a307fac30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009fd0 tx=0x7ff64c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195005b9440fefbd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.338+0000 7ff648ff9640 1 -- 172.21.15.103:0/540410063 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c018b60 con 0x560a307fa850 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.338+0000 7ff648ff9640 1 -- 172.21.15.103:0/540410063 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff64c016e20 con 0x560a307fa850 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff648ff9640 1 -- 172.21.15.103:0/540410063 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c017b10 con 0x560a307fa850 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65aa40640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff6400a2c00 0x7ff6400a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff64c01f000 tx=0x7ff64c022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65e87c740 1 -- 172.21.15.103:0/540410063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 msgr2=0x560a307fac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65e87c740 1 --2- 172.21.15.103:0/540410063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a307fac30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009fd0 tx=0x7ff64c009500 comp rx=0 tx=0).stop 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65e87c740 1 -- 172.21.15.103:0/540410063 shutdown_connections 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65e87c740 1 --2- 172.21.15.103:0/540410063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30807e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65e87c740 1 --2- 172.21.15.103:0/540410063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a308054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65e87c740 1 --2- 172.21.15.103:0/540410063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a307fac30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65e87c740 1 -- 172.21.15.103:0/540410063 >> 172.21.15.103:0/540410063 conn(0x560a307f6de0 msgr2=0x560a307f71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65e87c740 1 -- 172.21.15.103:0/540410063 shutdown_connections 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.339+0000 7ff65e87c740 1 -- 172.21.15.103:0/540410063 wait complete. 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.340+0000 7ff65e87c740 1 -- start start 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.340+0000 7ff65e87c740 1 Processor -- start 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.340+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a30802830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30802d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65a23f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a30802830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a308032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65e87c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a308086a0 con 0x560a307fa850 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff659a3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30802d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65a23f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a30802830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43604/0 (socket says 172.21.15.103:43604) 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff659a3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30802d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:60194/0 (socket says 172.21.15.103:60194) 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65a23f640 1 -- 172.21.15.103:0/508784561 learned_addr learned my addr 172.21.15.103:0/508784561 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a306e7900 con 0x560a307fb170 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560a30808820 con 0x560a30805a30 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65e87c740 10 client.0 _ll_register_callbacks cb 0x560a306e5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65aa40640 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a308032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65a23f640 1 -- 172.21.15.103:0/508784561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 msgr2=0x560a308032b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65a23f640 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a308032b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65a23f640 1 -- 172.21.15.103:0/508784561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 msgr2=0x560a30802d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65a23f640 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30802d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65a23f640 1 -- 172.21.15.103:0/508784561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a30727c50 con 0x560a307fa850 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff65a23f640 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a30802830 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff6500bc610 tx=0x7ff650009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ae22dcfe2ff582 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.341+0000 7ff63dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:01.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.342+0000 7ff63dffb640 1 -- 172.21.15.103:0/508784561 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6500bfb10 con 0x560a307fa850 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.342+0000 7ff63dffb640 1 -- 172.21.15.103:0/508784561 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff6500be4b0 con 0x560a307fa850 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.342+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64c016e20 con 0x560a307fa850 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.342+0000 7ff63dffb640 1 -- 172.21.15.103:0/508784561 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff6500c08b0 con 0x560a307fa850 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.342+0000 7ff65e87c740 10 client.15807 fetch_fsmap learned FSMap version 33 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.342+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560a308086a0 con 0x560a307fa850 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.342+0000 7ff63dffb640 1 -- 172.21.15.103:0/508784561 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6500d9190 con 0x560a307fa850 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.342+0000 7ff63dffb640 1 -- 172.21.15.103:0/508784561 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7ff650054060 con 0x560a307fa850 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.343+0000 7ff65e87c740 10 client.15807 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.343+0000 7ff65e87c740 10 client.15807 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.343+0000 7ff65e87c740 20 client.15807 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.343+0000 7ff65e87c740 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a3081dfe0 0x560a30820460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.343+0000 7ff65e87c740 4 client.15807 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:06:01.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.343+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff64c016e20 con 0x560a3081dfe0 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.343+0000 7ff659a3e640 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a3081dfe0 0x560a30820460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.343+0000 7ff659a3e640 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a3081dfe0 0x560a30820460 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97bc1eb9a01c354e server_cookie=98eefb88ffce3904 in_seq=0 out_seq=0 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.343+0000 7ff63dffb640 10 client.15807 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff63dffb640 1 -- 172.21.15.103:0/508784561 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff64c016e20 con 0x560a3081dfe0 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff63dffb640 10 client.15807 handle_command_reply: tid=0 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff65e87c740 1 client.15807 shutdown 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 msgr2=0x560a30802830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff65e87c740 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a30802830 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff6500bc610 tx=0x7ff650009630 comp rx=0 tx=0).stop 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 shutdown_connections 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff65e87c740 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a3081dfe0 0x560a30820460 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff65e87c740 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a308032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff65e87c740 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30802d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff65e87c740 1 --2- 172.21.15.103:0/508784561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fa850 0x560a30802830 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 >> 172.21.15.103:0/508784561 conn(0x560a307f6de0 msgr2=0x560a307fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.344+0000 7ff63dffb640 0 client.15807 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.345+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 shutdown_connections 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.345+0000 7ff65e87c740 1 -- 172.21.15.103:0/508784561 wait complete. 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.345+0000 7ff65e87c740 20 client.15807 trim_cache size 0 max 16384 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.371+0000 7ff65e87c740 1 Processor -- start 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.372+0000 7ff65e87c740 1 -- start start 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.372+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fb170 0x560a307fd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30825d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff65a23f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fb170 0x560a307fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff65a23f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fb170 0x560a307fd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43610/0 (socket says 172.21.15.103:43610) 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff65a23f640 1 -- 172.21.15.103:0/782192856 learned_addr learned my addr 172.21.15.103:0/782192856 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:01.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff659a3e640 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30825d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff65e87c740 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a30801810 0x560a30826270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff65e87c740 1 -- 172.21.15.103:0/782192856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541acbb0 con 0x560a307fb170 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff65e87c740 1 -- 172.21.15.103:0/782192856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541ae870 con 0x560a30801810 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff65e87c740 1 -- 172.21.15.103:0/782192856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541ad280 con 0x560a30805a30 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.373+0000 7ff65aa40640 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a30801810 0x560a30826270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.374+0000 7ff65aa40640 1 -- 172.21.15.103:0/782192856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 msgr2=0x560a30825d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.374+0000 7ff65aa40640 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30825d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.374+0000 7ff65aa40640 1 -- 172.21.15.103:0/782192856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fb170 msgr2=0x560a307fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.374+0000 7ff65aa40640 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fb170 0x560a307fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.374+0000 7ff65aa40640 1 -- 172.21.15.103:0/782192856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a307fafa0 con 0x560a30801810 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.375+0000 7ff65aa40640 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a30801810 0x560a30826270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009480 tx=0x7ff64c025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dbe80c5f4f8e82c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.375+0000 7ff63ffff640 1 -- 172.21.15.103:0/782192856 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c027070 con 0x560a30801810 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.375+0000 7ff63ffff640 1 -- 172.21.15.103:0/782192856 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff64c028040 con 0x560a30801810 2024-09-05T02:06:01.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.375+0000 7ff63ffff640 1 -- 172.21.15.103:0/782192856 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c017b50 con 0x560a30801810 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.377+0000 7ff65e87c740 1 -- 172.21.15.103:0/782192856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a30801810 msgr2=0x560a30826270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.377+0000 7ff65e87c740 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a30801810 0x560a30826270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009480 tx=0x7ff64c025000 comp rx=0 tx=0).stop 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.377+0000 7ff65e87c740 1 -- 172.21.15.103:0/782192856 shutdown_connections 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.377+0000 7ff65e87c740 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a30801810 0x560a30826270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.377+0000 7ff65e87c740 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30825d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.377+0000 7ff65e87c740 1 --2- 172.21.15.103:0/782192856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a307fb170 0x560a307fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.377+0000 7ff65e87c740 1 -- 172.21.15.103:0/782192856 >> 172.21.15.103:0/782192856 conn(0x560a307f6de0 msgr2=0x560a30801100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.378+0000 7ff65e87c740 1 -- 172.21.15.103:0/782192856 shutdown_connections 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.378+0000 7ff65e87c740 1 -- 172.21.15.103:0/782192856 wait complete. 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.379+0000 7ff65e87c740 1 -- start start 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.379+0000 7ff65e87c740 1 Processor -- start 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.380+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30812a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.380+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a30801810 0x560a30812f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.380+0000 7ff65e87c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30881d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.380+0000 7ff65e87c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541ae0e0 con 0x560a30801810 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.380+0000 7ff65e87c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541ad950 con 0x560a307fb170 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.380+0000 7ff65e87c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541aab90 con 0x560a30805a30 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.380+0000 7ff65e87c740 10 client.0 _ll_register_callbacks cb 0x7ff650005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.381+0000 7ff65a23f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30812a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.381+0000 7ff65a23f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30812a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:60210/0 (socket says 172.21.15.103:60210) 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.381+0000 7ff65a23f640 1 -- 172.21.15.103:0/854824597 learned_addr learned my addr 172.21.15.103:0/854824597 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.381+0000 7ff65aa40640 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30881d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.381+0000 7ff65aa40640 1 -- 172.21.15.103:0/854824597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 msgr2=0x560a30812a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.381+0000 7ff65aa40640 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30812a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.381+0000 7ff65aa40640 1 -- 172.21.15.103:0/854824597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a30801810 msgr2=0x560a30812f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.381+0000 7ff65aa40640 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a30801810 0x560a30812f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:01.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.381+0000 7ff65aa40640 1 -- 172.21.15.103:0/854824597 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a30882380 con 0x560a30805a30 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.382+0000 7ff65aa40640 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30881d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009480 tx=0x7ff64c017c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a44277c6fe842a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.382+0000 7ff63dffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.382+0000 7ff63dffb640 1 -- 172.21.15.103:0/854824597 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c027070 con 0x560a30805a30 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.382+0000 7ff63dffb640 1 -- 172.21.15.103:0/854824597 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff64c004bc0 con 0x560a30805a30 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.382+0000 7ff65e87c740 1 -- 172.21.15.103:0/854824597 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64c028040 con 0x560a30805a30 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.382+0000 7ff63dffb640 1 -- 172.21.15.103:0/854824597 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff64c004dc0 con 0x560a30805a30 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.383+0000 7ff65e87c740 10 client.25210 fetch_fsmap learned FSMap version 33 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.383+0000 7ff65e87c740 1 -- 172.21.15.103:0/854824597 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560a308896d0 con 0x560a30805a30 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.383+0000 7ff63dffb640 1 -- 172.21.15.103:0/854824597 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff64c0182b0 con 0x560a30805a30 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.383+0000 7ff63dffb640 1 -- 172.21.15.103:0/854824597 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7ff64c02d040 con 0x560a30805a30 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.384+0000 7ff65e87c740 10 client.25210 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.384+0000 7ff65e87c740 10 client.25210 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.384+0000 7ff65e87c740 20 client.25210 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.384+0000 7ff65e87c740 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a30889ca0 0x560a3088c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.384+0000 7ff65e87c740 4 client.25210 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}] 2024-09-05T02:06:01.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.384+0000 7ff65e87c740 1 -- 172.21.15.103:0/854824597 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}) -- 0x7ff64c028040 con 0x560a30889ca0 2024-09-05T02:06:01.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.384+0000 7ff65a23f640 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a30889ca0 0x560a3088c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:01.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.385+0000 7ff65a23f640 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a30889ca0 0x560a3088c0c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6774d53930e37fd7 server_cookie=2893d0f15ea6a73b in_seq=0 out_seq=0 2024-09-05T02:06:01.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:00.385+0000 7ff63dffb640 10 client.25210 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.403+0000 7ff63dffb640 1 -- 172.21.15.103:0/854824597 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7ff64c028040 con 0x560a30889ca0 2024-09-05T02:06:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.403+0000 7ff63dffb640 10 client.25210 handle_command_reply: tid=0 2024-09-05T02:06:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 client.25210 shutdown 2024-09-05T02:06:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 -- 172.21.15.103:0/854824597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 msgr2=0x560a30881d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30881d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009480 tx=0x7ff64c017c80 comp rx=0 tx=0).stop 2024-09-05T02:06:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 -- 172.21.15.103:0/854824597 shutdown_connections 2024-09-05T02:06:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x560a30889ca0 0x560a3088c0c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff63dffb640 0 client.25210 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a30805a30 0x560a30881d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a30801810 0x560a30812f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 --2- 172.21.15.103:0/854824597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a307fb170 0x560a30812a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 -- 172.21.15.103:0/854824597 >> 172.21.15.103:0/854824597 conn(0x560a307f6de0 msgr2=0x560a308027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 -- 172.21.15.103:0/854824597 shutdown_connections 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 1 -- 172.21.15.103:0/854824597 wait complete. 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.404+0000 7ff65e87c740 20 client.25210 trim_cache size 0 max 16384 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 33, 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:06:02.407 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 10, 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "f3111ecb": { 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 10, 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 141.8, 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:06:02.408 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:06:02.409 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:06:02.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:06:02.409 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:06:02.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:02.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:02.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:02.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:02.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:02.409 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff6400a2c00 msgr2=0x7ff6400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff6400a2c00 0x7ff6400a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff64c01f000 tx=0x7ff64c022000 comp rx=0 tx=0).stop 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654100a80 msgr2=0x7ff6541961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654100a80 0x7ff6541961c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff650005ab0 tx=0x7ff650004a90 comp rx=0 tx=0).stop 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3680144533 shutdown_connections 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff6400a2c00 0x7ff6400a50c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff654106a80 0x7ff65419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff654101420 0x7ff654196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 --2- 172.21.15.103:0/3680144533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff654100a80 0x7ff6541961c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3680144533 >> 172.21.15.103:0/3680144533 conn(0x7ff6540fc7a0 msgr2=0x7ff6540fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:02.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3680144533 shutdown_connections 2024-09-05T02:06:02.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:02.405+0000 7ff65c4a3640 1 -- 172.21.15.103:0/3680144533 wait complete. 2024-09-05T02:06:02.549 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:02 smithi103 ceph-mon[30902]: pgmap v458: 129 pgs: 129 active+clean; 212 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 301 op/s 2024-09-05T02:06:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:02 smithi167 ceph-mon[35122]: pgmap v458: 129 pgs: 129 active+clean; 212 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 301 op/s 2024-09-05T02:06:02.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:02 smithi139 ceph-mon[34262]: pgmap v458: 129 pgs: 129 active+clean; 212 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 301 op/s 2024-09-05T02:06:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.748+0000 7fa04674b640 1 -- 172.21.15.103:0/3691433757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040107880 msgr2=0x7fa040109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.748+0000 7fa04674b640 1 --2- 172.21.15.103:0/3691433757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040107880 0x7fa040109c90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa038009ca0 tx=0x7fa03802a8c0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.748+0000 7fa04674b640 1 -- 172.21.15.103:0/3691433757 shutdown_connections 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.748+0000 7fa04674b640 1 --2- 172.21.15.103:0/3691433757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040107880 0x7fa040109c90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.748+0000 7fa04674b640 1 --2- 172.21.15.103:0/3691433757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa040104f50 0x7fa040107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.748+0000 7fa04674b640 1 --2- 172.21.15.103:0/3691433757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa040102620 0x7fa040104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.748+0000 7fa04674b640 1 -- 172.21.15.103:0/3691433757 >> 172.21.15.103:0/3691433757 conn(0x7fa0400fc1e0 msgr2=0x7fa0400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.748+0000 7fa04674b640 1 -- 172.21.15.103:0/3691433757 shutdown_connections 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.748+0000 7fa04674b640 1 -- 172.21.15.103:0/3691433757 wait complete. 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa04674b640 1 Processor -- start 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa04674b640 1 -- start start 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa04674b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa040102620 0x7fa040197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa04674b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040104f50 0x7fa040198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa04674b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa040107880 0x7fa04019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa04674b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa04010d6b0 con 0x7fa040104f50 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa04674b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa04010d530 con 0x7fa040102620 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa04674b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa04010d830 con 0x7fa040107880 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa03f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040104f50 0x7fa040198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa03f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040104f50 0x7fa040198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43636/0 (socket says 172.21.15.103:43636) 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa03f7fe640 1 -- 172.21.15.103:0/3998126786 learned_addr learned my addr 172.21.15.103:0/3998126786 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa03ffff640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa040102620 0x7fa040197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.749+0000 7fa044ce8640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa040107880 0x7fa04019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa03f7fe640 1 -- 172.21.15.103:0/3998126786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa040107880 msgr2=0x7fa04019f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa03f7fe640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa040107880 0x7fa04019f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa03f7fe640 1 -- 172.21.15.103:0/3998126786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa040102620 msgr2=0x7fa040197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa03f7fe640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa040102620 0x7fa040197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa03f7fe640 1 -- 172.21.15.103:0/3998126786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04019f960 con 0x7fa040104f50 2024-09-05T02:06:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa03f7fe640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040104f50 0x7fa040198280 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa03400d770 tx=0x7fa03400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa03d7fa640 1 -- 172.21.15.103:0/3998126786 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034017070 con 0x7fa040104f50 2024-09-05T02:06:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa03d7fa640 1 -- 172.21.15.103:0/3998126786 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa034009c30 con 0x7fa040104f50 2024-09-05T02:06:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa04674b640 1 -- 172.21.15.103:0/3998126786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04019fc50 con 0x7fa040104f50 2024-09-05T02:06:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa03d7fa640 1 -- 172.21.15.103:0/3998126786 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03400f620 con 0x7fa040104f50 2024-09-05T02:06:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.750+0000 7fa04674b640 1 -- 172.21.15.103:0/3998126786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0401a01a0 con 0x7fa040104f50 2024-09-05T02:06:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.752+0000 7fa03d7fa640 1 -- 172.21.15.103:0/3998126786 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa03401a070 con 0x7fa040104f50 2024-09-05T02:06:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa0495c6740 1 Processor -- start 2024-09-05T02:06:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa0495c6740 1 -- start start 2024-09-05T02:06:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 0x5599db6c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa0495c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5599db6292f0 con 0x5599db6cb370 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa0495c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5599db5d1270 con 0x5599db6c59a0 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa0495c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5599db6252b0 con 0x5599db6c62e0 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03d7fa640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa0280a2c30 0x7fa0280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa044ce8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 0x5599db6c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43648/0 (socket says 172.21.15.103:43648) 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03f7fe640 1 -- 172.21.15.103:0/3049588132 learned_addr learned my addr 172.21.15.103:0/3049588132 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:46198/0 (socket says 172.21.15.103:46198) 2024-09-05T02:06:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa044ce8640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa0280a2c30 0x7fa0280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03ffff640 1 -- 172.21.15.103:0/3049588132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 msgr2=0x5599db6c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03ffff640 1 --2- 172.21.15.103:0/3049588132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 0x5599db6c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03ffff640 1 -- 172.21.15.103:0/3049588132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 msgr2=0x5599db6cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03ffff640 1 --2- 172.21.15.103:0/3049588132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03d7fa640 1 -- 172.21.15.103:0/3998126786 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa0340cee70 con 0x7fa040104f50 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.753+0000 7fa03ffff640 1 -- 172.21.15.103:0/3049588132 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599db5d0340 con 0x5599db6c62e0 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa03ffff640 1 --2- 172.21.15.103:0/3049588132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6cada0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa030004630 tx=0x7fa03000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cce46d999e209d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa044ce8640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa0280a2c30 0x7fa0280a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa038006010 tx=0x7fa03802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0267fc640 1 -- 172.21.15.103:0/3049588132 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030003bd0 con 0x5599db6c62e0 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0267fc640 1 -- 172.21.15.103:0/3049588132 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa030015070 con 0x5599db6c62e0 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0267fc640 1 -- 172.21.15.103:0/3049588132 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030004d10 con 0x5599db6c62e0 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0495c6740 1 -- 172.21.15.103:0/3049588132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 msgr2=0x5599db6cada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0495c6740 1 --2- 172.21.15.103:0/3049588132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6cada0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa030004630 tx=0x7fa03000bb20 comp rx=0 tx=0).stop 2024-09-05T02:06:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0495c6740 1 -- 172.21.15.103:0/3049588132 shutdown_connections 2024-09-05T02:06:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0495c6740 1 --2- 172.21.15.103:0/3049588132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0495c6740 1 --2- 172.21.15.103:0/3049588132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6cada0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0495c6740 1 --2- 172.21.15.103:0/3049588132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 0x5599db6c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0495c6740 1 -- 172.21.15.103:0/3049588132 >> 172.21.15.103:0/3049588132 conn(0x5599db6c1010 msgr2=0x5599db6c1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0495c6740 1 -- 172.21.15.103:0/3049588132 shutdown_connections 2024-09-05T02:06:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.754+0000 7fa0495c6740 1 -- 172.21.15.103:0/3049588132 wait complete. 2024-09-05T02:06:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa0495c6740 1 -- start start 2024-09-05T02:06:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa0495c6740 1 Processor -- start 2024-09-05T02:06:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 0x5599db6cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 0x5599db6d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa0495c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5599db6ce340 con 0x5599db6cb370 2024-09-05T02:06:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa0495c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5599db5b2290 con 0x5599db6c62e0 2024-09-05T02:06:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa0495c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5599db6ce4c0 con 0x5599db6c59a0 2024-09-05T02:06:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa0495c6740 10 client.0 _ll_register_callbacks cb 0x5599db5e3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa044ce8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa044ce8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43654/0 (socket says 172.21.15.103:43654) 2024-09-05T02:06:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa044ce8640 1 -- 172.21.15.103:0/1139881913 learned_addr learned my addr 172.21.15.103:0/1139881913 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa03f7fe640 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 0x5599db6d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.755+0000 7fa03ffff640 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 0x5599db6cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa044ce8640 1 -- 172.21.15.103:0/1139881913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 msgr2=0x5599db6cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa044ce8640 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 0x5599db6cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa044ce8640 1 -- 172.21.15.103:0/1139881913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 msgr2=0x5599db6d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa044ce8640 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 0x5599db6d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa044ce8640 1 -- 172.21.15.103:0/1139881913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599db33c210 con 0x5599db6cb370 2024-09-05T02:06:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa044ce8640 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d7140 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa038009ca0 tx=0x7fa0380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8335c19c6f9f1d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa01f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa01f7fe640 1 -- 172.21.15.103:0/1139881913 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03804f030 con 0x5599db6cb370 2024-09-05T02:06:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa01f7fe640 1 -- 172.21.15.103:0/1139881913 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa038004510 con 0x5599db6cb370 2024-09-05T02:06:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa0495c6740 1 -- 172.21.15.103:0/1139881913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa030015070 con 0x5599db6cb370 2024-09-05T02:06:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa01f7fe640 1 -- 172.21.15.103:0/1139881913 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa038004710 con 0x5599db6cb370 2024-09-05T02:06:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa0495c6740 10 client.15822 fetch_fsmap learned FSMap version 33 2024-09-05T02:06:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa0495c6740 1 -- 172.21.15.103:0/1139881913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5599db408620 con 0x5599db6cb370 2024-09-05T02:06:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa01f7fe640 1 -- 172.21.15.103:0/1139881913 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa030015070 con 0x5599db6cb370 2024-09-05T02:06:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa01f7fe640 1 -- 172.21.15.103:0/1139881913 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fa03804a040 con 0x5599db6cb370 2024-09-05T02:06:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa0495c6740 10 client.15822 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:06:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa0495c6740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.756+0000 7fa0495c6740 20 client.15822 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.757+0000 7fa0495c6740 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5599db6db8b0 0x5599db6ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.757+0000 7fa0495c6740 4 client.15822 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:06:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.757+0000 7fa0495c6740 1 -- 172.21.15.103:0/1139881913 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599db636a20 con 0x5599db6db8b0 2024-09-05T02:06:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.757+0000 7fa03ffff640 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5599db6db8b0 0x5599db6ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.757+0000 7fa03ffff640 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5599db6db8b0 0x5599db6ddd30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e663ed4fbb578fe server_cookie=21b1a263a372811f in_seq=0 out_seq=0 2024-09-05T02:06:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.757+0000 7fa01f7fe640 10 client.15822 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa01f7fe640 1 -- 172.21.15.103:0/1139881913 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599db636a20 con 0x5599db6db8b0 2024-09-05T02:06:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa01f7fe640 10 client.15822 handle_command_reply: tid=0 2024-09-05T02:06:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 client.15822 shutdown 2024-09-05T02:06:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 -- 172.21.15.103:0/1139881913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 msgr2=0x5599db6d7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d7140 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa038009ca0 tx=0x7fa0380095c0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 -- 172.21.15.103:0/1139881913 shutdown_connections 2024-09-05T02:06:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5599db6db8b0 0x5599db6ddd30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d7140 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa01f7fe640 0 client.15822 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 0x5599db6d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 --2- 172.21.15.103:0/1139881913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 0x5599db6cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 -- 172.21.15.103:0/1139881913 >> 172.21.15.103:0/1139881913 conn(0x5599db6c1010 msgr2=0x5599db6c2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 -- 172.21.15.103:0/1139881913 shutdown_connections 2024-09-05T02:06:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 1 -- 172.21.15.103:0/1139881913 wait complete. 2024-09-05T02:06:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.758+0000 7fa0495c6740 20 client.15822 trim_cache size 0 max 16384 2024-09-05T02:06:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa0495c6740 1 Processor -- start 2024-09-05T02:06:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa0495c6740 1 -- start start 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 0x5599db6c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 0x5599db6d70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa0495c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401aa700 con 0x5599db6cb370 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa0495c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401ac3c0 con 0x5599db6c62e0 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa0495c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401aadd0 con 0x5599db6c59a0 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa044ce8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.772+0000 7fa044ce8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6e25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43656/0 (socket says 172.21.15.103:43656) 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa044ce8640 1 -- 172.21.15.103:0/3394318265 learned_addr learned my addr 172.21.15.103:0/3394318265 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa03f7fe640 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 0x5599db6d70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa03ffff640 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 0x5599db6c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa044ce8640 1 -- 172.21.15.103:0/3394318265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 msgr2=0x5599db6c00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa044ce8640 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 0x5599db6c00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa044ce8640 1 -- 172.21.15.103:0/3394318265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 msgr2=0x5599db6d70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa044ce8640 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 0x5599db6d70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa044ce8640 1 -- 172.21.15.103:0/3394318265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599db6c67c0 con 0x5599db6cb370 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa03ffff640 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 0x5599db6c00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa044ce8640 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6e25f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa038009930 tx=0x7fa038005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2158e0bf9eb39b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa025ffb640 1 -- 172.21.15.103:0/3394318265 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03804f070 con 0x5599db6cb370 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa025ffb640 1 -- 172.21.15.103:0/3394318265 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa038036050 con 0x5599db6cb370 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa025ffb640 1 -- 172.21.15.103:0/3394318265 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038030b80 con 0x5599db6cb370 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.773+0000 7fa0495c6740 1 -- 172.21.15.103:0/3394318265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 msgr2=0x5599db6e25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6e25f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa038009930 tx=0x7fa038005f30 comp rx=0 tx=0).stop 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 -- 172.21.15.103:0/3394318265 shutdown_connections 2024-09-05T02:06:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6e25f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c62e0 0x5599db6d70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 --2- 172.21.15.103:0/3394318265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c59a0 0x5599db6c00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 -- 172.21.15.103:0/3394318265 >> 172.21.15.103:0/3394318265 conn(0x5599db6c1010 msgr2=0x5599db6bf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 -- 172.21.15.103:0/3394318265 shutdown_connections 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 -- 172.21.15.103:0/3394318265 wait complete. 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 -- start start 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 Processor -- start 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 0x5599db6d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401abc30 con 0x5599db6cb370 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401ab4a0 con 0x5599db6c59a0 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401a86e0 con 0x5599db6c62e0 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa0495c6740 10 client.0 _ll_register_callbacks cb 0x7fa038009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa044ce8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa03f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa044ce8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43664/0 (socket says 172.21.15.103:43664) 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.774+0000 7fa044ce8640 1 -- 172.21.15.103:0/4275629659 learned_addr learned my addr 172.21.15.103:0/4275629659 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa03ffff640 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 0x5599db6d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa03f7fe640 1 -- 172.21.15.103:0/4275629659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 msgr2=0x5599db6d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa03f7fe640 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 0x5599db6d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa03f7fe640 1 -- 172.21.15.103:0/4275629659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 msgr2=0x5599db6d6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa03f7fe640 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa03f7fe640 1 -- 172.21.15.103:0/4275629659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599db6d6a60 con 0x5599db6c62e0 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa044ce8640 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa03f7fe640 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6d5490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa03409b360 tx=0x7fa03409af10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a72d481681aa06 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa01f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa01f7fe640 1 -- 172.21.15.103:0/4275629659 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034021040 con 0x5599db6c62e0 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa01f7fe640 1 -- 172.21.15.103:0/4275629659 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa0340b4260 con 0x5599db6c62e0 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa0495c6740 1 -- 172.21.15.103:0/4275629659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa038036050 con 0x5599db6c62e0 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa01f7fe640 1 -- 172.21.15.103:0/4275629659 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa034005ab0 con 0x5599db6c62e0 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa0495c6740 10 client.25225 fetch_fsmap learned FSMap version 33 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa0495c6740 1 -- 172.21.15.103:0/4275629659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5599db746c60 con 0x5599db6c62e0 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa01f7fe640 1 -- 172.21.15.103:0/4275629659 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa038036050 con 0x5599db6c62e0 2024-09-05T02:06:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.775+0000 7fa01f7fe640 1 -- 172.21.15.103:0/4275629659 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fa03409f170 con 0x5599db6c62e0 2024-09-05T02:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.776+0000 7fa0495c6740 10 client.25225 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T02:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.776+0000 7fa0495c6740 10 client.25225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.776+0000 7fa0495c6740 20 client.25225 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.776+0000 7fa0495c6740 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5599db747650 0x5599db749a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.776+0000 7fa0495c6740 4 client.25225 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.776+0000 7fa0495c6740 1 -- 172.21.15.103:0/4275629659 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599db630d70 con 0x5599db747650 2024-09-05T02:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.776+0000 7fa03ffff640 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5599db747650 0x5599db749a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.776+0000 7fa01f7fe640 10 client.25225 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.776+0000 7fa03ffff640 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5599db747650 0x5599db749a70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78b08eb13c49492 server_cookie=7fb4071609aff26 in_seq=0 out_seq=0 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa01f7fe640 1 -- 172.21.15.103:0/4275629659 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599db630d70 con 0x5599db747650 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa01f7fe640 10 client.25225 handle_command_reply: tid=0 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa0495c6740 1 client.25225 shutdown 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa0495c6740 1 -- 172.21.15.103:0/4275629659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 msgr2=0x5599db6d5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa0495c6740 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6d5490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa03409b360 tx=0x7fa03409af10 comp rx=0 tx=0).stop 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa0495c6740 1 -- 172.21.15.103:0/4275629659 shutdown_connections 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa0495c6740 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5599db747650 0x5599db749a70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa0495c6740 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5599db6cb370 0x5599db6d6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa01f7fe640 0 client.25225 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa0495c6740 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599db6c62e0 0x5599db6d5490 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa0495c6740 1 --2- 172.21.15.103:0/4275629659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599db6c59a0 0x5599db6d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.880+0000 7fa0495c6740 1 -- 172.21.15.103:0/4275629659 >> 172.21.15.103:0/4275629659 conn(0x5599db6c1010 msgr2=0x5599db6c1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa0495c6740 1 -- 172.21.15.103:0/4275629659 shutdown_connections 2024-09-05T02:06:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa0495c6740 1 -- 172.21.15.103:0/4275629659 wait complete. 2024-09-05T02:06:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa0495c6740 20 client.25225 trim_cache size 0 max 16384 2024-09-05T02:06:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:06:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:06:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:06:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:06:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 -- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa0280a2c30 msgr2=0x7fa0280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa0280a2c30 0x7fa0280a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa038006010 tx=0x7fa03802a8c0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 -- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040104f50 msgr2=0x7fa040198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040104f50 0x7fa040198280 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa03400d770 tx=0x7fa03400dc30 comp rx=0 tx=0).stop 2024-09-05T02:06:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 -- 172.21.15.103:0/3998126786 shutdown_connections 2024-09-05T02:06:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa0280a2c30 0x7fa0280a50f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa040107880 0x7fa04019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa040104f50 0x7fa040198280 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 --2- 172.21.15.103:0/3998126786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa040102620 0x7fa040197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.881+0000 7fa04674b640 1 -- 172.21.15.103:0/3998126786 >> 172.21.15.103:0/3998126786 conn(0x7fa0400fc1e0 msgr2=0x7fa040102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.882+0000 7fa04674b640 1 -- 172.21.15.103:0/3998126786 shutdown_connections 2024-09-05T02:06:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:02.882+0000 7fa04674b640 1 -- 172.21.15.103:0/3998126786 wait complete. 2024-09-05T02:06:03.874 INFO:tasks.workunit.client.0.smithi103.stdout: 46 8881 109 5149 113 2638 194 2024-09-05T02:06:04.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:04 smithi103 ceph-mon[30902]: pgmap v459: 129 pgs: 129 active+clean; 217 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 328 op/s 2024-09-05T02:06:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:04 smithi167 ceph-mon[35122]: pgmap v459: 129 pgs: 129 active+clean; 217 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 328 op/s 2024-09-05T02:06:04.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:04 smithi139 ceph-mon[34262]: pgmap v459: 129 pgs: 129 active+clean; 217 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 328 op/s 2024-09-05T02:06:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:06 smithi167 ceph-mon[35122]: pgmap v460: 129 pgs: 129 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 361 op/s 2024-09-05T02:06:06.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:06 smithi139 ceph-mon[34262]: pgmap v460: 129 pgs: 129 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 361 op/s 2024-09-05T02:06:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:06 smithi103 ceph-mon[30902]: pgmap v460: 129 pgs: 129 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 361 op/s 2024-09-05T02:06:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:08 smithi167 ceph-mon[35122]: pgmap v461: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.5 MiB/s wr, 294 op/s 2024-09-05T02:06:08.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:08 smithi139 ceph-mon[34262]: pgmap v461: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.5 MiB/s wr, 294 op/s 2024-09-05T02:06:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:08 smithi103 ceph-mon[30902]: pgmap v461: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.5 MiB/s wr, 294 op/s 2024-09-05T02:06:10.189 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f3111ecb', quiesce took 14.5 seconds. Will release after: 7.25 2024-09-05T02:06:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:10 smithi167 ceph-mon[35122]: pgmap v462: 129 pgs: 129 active+clean; 217 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 317 op/s 2024-09-05T02:06:10.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:10 smithi139 ceph-mon[34262]: pgmap v462: 129 pgs: 129 active+clean; 217 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 317 op/s 2024-09-05T02:06:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:10 smithi103 ceph-mon[30902]: pgmap v462: 129 pgs: 129 active+clean; 217 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 317 op/s 2024-09-05T02:06:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:12 smithi167 ceph-mon[35122]: pgmap v463: 129 pgs: 129 active+clean; 217 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.2 MiB/s wr, 276 op/s 2024-09-05T02:06:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:12 smithi139 ceph-mon[34262]: pgmap v463: 129 pgs: 129 active+clean; 217 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.2 MiB/s wr, 276 op/s 2024-09-05T02:06:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:12 smithi103 ceph-mon[30902]: pgmap v463: 129 pgs: 129 active+clean; 217 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.2 MiB/s wr, 276 op/s 2024-09-05T02:06:13.875 INFO:tasks.workunit.client.0.smithi103.stdout: 46 12785 0 6710 0 3709 0 2024-09-05T02:06:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:14 smithi167 ceph-mon[35122]: pgmap v464: 129 pgs: 129 active+clean; 218 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1003 KiB/s wr, 247 op/s 2024-09-05T02:06:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:14 smithi139 ceph-mon[34262]: pgmap v464: 129 pgs: 129 active+clean; 218 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1003 KiB/s wr, 247 op/s 2024-09-05T02:06:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:14 smithi103 ceph-mon[30902]: pgmap v464: 129 pgs: 129 active+clean; 218 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1003 KiB/s wr, 247 op/s 2024-09-05T02:06:15.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:15 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:06:15.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:15 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:06:15.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:15 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:06:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.004+0000 7f2b0fe23640 1 -- 172.21.15.103:0/550093202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b08100320 msgr2=0x7f2b08100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.004+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/550093202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b08100320 0x7f2b08100720 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400a910 tx=0x7f2b0402a610 comp rx=0 tx=0).stop 2024-09-05T02:06:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.004+0000 7f2b0fe23640 1 -- 172.21.15.103:0/550093202 shutdown_connections 2024-09-05T02:06:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.004+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/550093202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b08101d70 0x7f2b08108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.004+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/550093202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b081013d0 0x7f2b08101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.004+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/550093202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b08100320 0x7f2b08100720 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.004+0000 7f2b0fe23640 1 -- 172.21.15.103:0/550093202 >> 172.21.15.103:0/550093202 conn(0x7f2b080fb500 msgr2=0x7f2b080fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.004+0000 7f2b0fe23640 1 -- 172.21.15.103:0/550093202 shutdown_connections 2024-09-05T02:06:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.004+0000 7f2b0fe23640 1 -- 172.21.15.103:0/550093202 wait complete. 2024-09-05T02:06:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0fe23640 1 Processor -- start 2024-09-05T02:06:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0fe23640 1 -- start start 2024-09-05T02:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0fe23640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b08100320 0x7f2b08195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0fe23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b081013d0 0x7f2b08195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0fe23640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b08101d70 0x7f2b0819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0dbbf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b08100320 0x7f2b08195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b081013d0 0x7f2b08195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0dbbf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b08100320 0x7f2b08195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:43686/0 (socket says 172.21.15.103:43686) 2024-09-05T02:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b081013d0 0x7f2b08195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:39154/0 (socket says 172.21.15.103:39154) 2024-09-05T02:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0dbbf640 1 -- 172.21.15.103:0/1337410460 learned_addr learned my addr 172.21.15.103:0/1337410460 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0810b470 con 0x7f2b08101d70 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0810b2f0 con 0x7f2b081013d0 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0810b5f0 con 0x7f2b08100320 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.005+0000 7f2b0e3c0640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b08101d70 0x7f2b0819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0e3c0640 1 -- 172.21.15.103:0/1337410460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b08100320 msgr2=0x7f2b08195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0e3c0640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b08100320 0x7f2b08195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0e3c0640 1 -- 172.21.15.103:0/1337410460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b081013d0 msgr2=0x7f2b08195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0e3c0640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b081013d0 0x7f2b08195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0e3c0640 1 -- 172.21.15.103:0/1337410460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b081a1ad0 con 0x7f2b08101d70 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b081013d0 0x7f2b08195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b08100320 0x7f2b08195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0e3c0640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b08101d70 0x7f2b0819d050 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009fd0 tx=0x7f2b0000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2afeffd640 1 -- 172.21.15.103:0/1337410460 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000ce20 con 0x7f2b08101d70 2024-09-05T02:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b081a1dc0 con 0x7f2b08101d70 2024-09-05T02:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2afeffd640 1 -- 172.21.15.103:0/1337410460 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b00005020 con 0x7f2b08101d70 2024-09-05T02:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2afeffd640 1 -- 172.21.15.103:0/1337410460 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b00010690 con 0x7f2b08101d70 2024-09-05T02:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.006+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b081a22e0 con 0x7f2b08101d70 2024-09-05T02:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.010+0000 7f2afeffd640 1 -- 172.21.15.103:0/1337410460 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2b000051d0 con 0x7f2b08101d70 2024-09-05T02:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.010+0000 7f2b12bbe740 1 Processor -- start 2024-09-05T02:06:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.010+0000 7f2b12bbe740 1 -- start start 2024-09-05T02:06:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.010+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.010+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 0x5644be6bdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.010+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.010+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5644be61c2f0 con 0x5644be6be370 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.010+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5644be5c4270 con 0x5644be6b89a0 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.010+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5644be6182b0 con 0x5644be6b92e0 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54900/0 (socket says 172.21.15.103:54900) 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0d3be640 1 -- 172.21.15.103:0/3409970262 learned_addr learned my addr 172.21.15.103:0/3409970262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/3409970262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 0x5644be6bdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2afeffd640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2af40a2c30 0x7f2af40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0d3be640 1 -- 172.21.15.103:0/3409970262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 msgr2=0x5644be6bdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/3409970262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 0x5644be6bdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0d3be640 1 -- 172.21.15.103:0/3409970262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 msgr2=0x5644be6b8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/3409970262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6b8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0d3be640 1 -- 172.21.15.103:0/3409970262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644be5c3340 con 0x5644be6be370 2024-09-05T02:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2b0e3c0640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2af40a2c30 0x7f2af40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.011+0000 7f2afeffd640 1 -- 172.21.15.103:0/1337410460 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2b000cfae0 con 0x7f2b08101d70 2024-09-05T02:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.012+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/3409970262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c0830 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2af800b800 tx=0x7f2af8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ac2777e5f68c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.012+0000 7f2af3fff640 1 -- 172.21.15.103:0/3409970262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8016d50 con 0x5644be6be370 2024-09-05T02:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.012+0000 7f2b0e3c0640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2af40a2c30 0x7f2af40a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b00098390 tx=0x7f2b00013b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.012+0000 7f2af3fff640 1 -- 172.21.15.103:0/3409970262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2af8004050 con 0x5644be6be370 2024-09-05T02:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.012+0000 7f2af3fff640 1 -- 172.21.15.103:0/3409970262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8005280 con 0x5644be6be370 2024-09-05T02:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3409970262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 msgr2=0x5644be6c0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3409970262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c0830 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2af800b800 tx=0x7f2af8005f90 comp rx=0 tx=0).stop 2024-09-05T02:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3409970262 shutdown_connections 2024-09-05T02:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3409970262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c0830 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3409970262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 0x5644be6bdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3409970262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3409970262 >> 172.21.15.103:0/3409970262 conn(0x5644be6b4010 msgr2=0x5644be6b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3409970262 shutdown_connections 2024-09-05T02:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3409970262 wait complete. 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 -- start start 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.013+0000 7f2b12bbe740 1 Processor -- start 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6b89a0 0x5644be6c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6be370 0x5644be6ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5644be6c12e0 con 0x5644be6b89a0 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5644be5a5290 con 0x5644be6b92e0 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5644be6c1460 con 0x5644be6be370 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6b89a0 0x5644be6c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b12bbe740 10 client.0 _ll_register_callbacks cb 0x5644be5d6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6b89a0 0x5644be6c2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54914/0 (socket says 172.21.15.103:54914) 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b0d3be640 1 -- 172.21.15.103:0/4113542214 learned_addr learned my addr 172.21.15.103:0/4113542214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b0d3be640 1 -- 172.21.15.103:0/4113542214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6be370 msgr2=0x5644be6ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6be370 0x5644be6ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b0d3be640 1 -- 172.21.15.103:0/4113542214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 msgr2=0x5644be6c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.014+0000 7f2b0d3be640 1 -- 172.21.15.103:0/4113542214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644be32f210 con 0x5644be6b89a0 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6b89a0 0x5644be6c2b60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2af800b5c0 tx=0x7f2af8004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ea76aae681e0aa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2af17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2af17fa640 1 -- 172.21.15.103:0/4113542214 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af802a020 con 0x5644be6b89a0 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2af17fa640 1 -- 172.21.15.103:0/4113542214 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2af8018d40 con 0x5644be6b89a0 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2b12bbe740 1 -- 172.21.15.103:0/4113542214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2af8004050 con 0x5644be6b89a0 2024-09-05T02:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2af17fa640 1 -- 172.21.15.103:0/4113542214 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2af8035050 con 0x5644be6b89a0 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2b12bbe740 10 client.15840 fetch_fsmap learned FSMap version 34 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2b12bbe740 1 -- 172.21.15.103:0/4113542214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5644be3fb620 con 0x5644be6b89a0 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.015+0000 7f2af17fa640 1 -- 172.21.15.103:0/4113542214 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2af8004050 con 0x5644be6b89a0 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2af17fa640 1 -- 172.21.15.103:0/4113542214 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7f2af8004ca0 con 0x5644be6b89a0 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2b12bbe740 10 client.15840 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2b12bbe740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2b12bbe740 20 client.15840 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5644be6ce4f0 0x5644be6d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2b12bbe740 4 client.15840 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2b12bbe740 1 -- 172.21.15.103:0/4113542214 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644be629a20 con 0x5644be6ce4f0 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5644be6ce4f0 0x5644be6d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2af17fa640 10 client.15840 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.016+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5644be6ce4f0 0x5644be6d08e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f01b53ef63f5b52 server_cookie=eacc1cedd1d1a821 in_seq=0 out_seq=0 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2af17fa640 1 -- 172.21.15.103:0/4113542214 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644be629a20 con 0x5644be6ce4f0 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2af17fa640 10 client.15840 handle_command_reply: tid=0 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 client.15840 shutdown 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 -- 172.21.15.103:0/4113542214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6b89a0 msgr2=0x5644be6c2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6b89a0 0x5644be6c2b60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2af800b5c0 tx=0x7f2af8004200 comp rx=0 tx=0).stop 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 -- 172.21.15.103:0/4113542214 shutdown_connections 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5644be6ce4f0 0x5644be6d08e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6be370 0x5644be6ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2af17fa640 0 client.15840 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/4113542214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6b89a0 0x5644be6c2b60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 -- 172.21.15.103:0/4113542214 >> 172.21.15.103:0/4113542214 conn(0x5644be6b4010 msgr2=0x5644be6b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 -- 172.21.15.103:0/4113542214 shutdown_connections 2024-09-05T02:06:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.017+0000 7f2b12bbe740 1 -- 172.21.15.103:0/4113542214 wait complete. 2024-09-05T02:06:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.018+0000 7f2b12bbe740 20 client.15840 trim_cache size 0 max 16384 2024-09-05T02:06:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b12bbe740 1 Processor -- start 2024-09-05T02:06:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b12bbe740 1 -- start start 2024-09-05T02:06:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b89a0 0x5644be6c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081ac880 con 0x5644be6be370 2024-09-05T02:06:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081ae4b0 con 0x5644be6b92e0 2024-09-05T02:06:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081acf50 con 0x5644be6b89a0 2024-09-05T02:06:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b0dbbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b0dbbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:39186/0 (socket says 172.21.15.103:39186) 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b0dbbf640 1 -- 172.21.15.103:0/1943636897 learned_addr learned my addr 172.21.15.103:0/1943636897 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.045+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/1943636897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b89a0 0x5644be6c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.046+0000 7f2b0dbbf640 1 -- 172.21.15.103:0/1943636897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b89a0 msgr2=0x5644be6c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.046+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/1943636897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b89a0 0x5644be6c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.046+0000 7f2b0dbbf640 1 -- 172.21.15.103:0/1943636897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 msgr2=0x5644be6c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.046+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/1943636897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.046+0000 7f2b0dbbf640 1 -- 172.21.15.103:0/1943636897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644be6bd720 con 0x5644be6b92e0 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.046+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/1943636897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6ca0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b04000c00 tx=0x7f2b0402f900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef86b39c2adc86ec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.046+0000 7f2af37fe640 1 -- 172.21.15.103:0/1943636897 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04035040 con 0x5644be6b92e0 2024-09-05T02:06:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.046+0000 7f2af37fe640 1 -- 172.21.15.103:0/1943636897 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b040498e0 con 0x5644be6b92e0 2024-09-05T02:06:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.046+0000 7f2af37fe640 1 -- 172.21.15.103:0/1943636897 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0403d9d0 con 0x5644be6b92e0 2024-09-05T02:06:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 -- 172.21.15.103:0/1943636897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 msgr2=0x5644be6ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/1943636897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6ca0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b04000c00 tx=0x7f2b0402f900 comp rx=0 tx=0).stop 2024-09-05T02:06:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 -- 172.21.15.103:0/1943636897 shutdown_connections 2024-09-05T02:06:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/1943636897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be6c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/1943636897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b92e0 0x5644be6ca0a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/1943636897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b89a0 0x5644be6c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 -- 172.21.15.103:0/1943636897 >> 172.21.15.103:0/1943636897 conn(0x5644be6b4010 msgr2=0x5644be6ba480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 -- 172.21.15.103:0/1943636897 shutdown_connections 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 -- 172.21.15.103:0/1943636897 wait complete. 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 -- start start 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.047+0000 7f2b12bbe740 1 Processor -- start 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 0x5644be6c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b12bbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be7370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081addb0 con 0x5644be6be370 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081ad620 con 0x5644be6b89a0 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b12bbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081aa860 con 0x5644be6b92e0 2024-09-05T02:06:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b12bbe740 10 client.0 _ll_register_callbacks cb 0x7f2af8016530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0d3be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:39190/0 (socket says 172.21.15.103:39190) 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0d3be640 1 -- 172.21.15.103:0/3170074714 learned_addr learned my addr 172.21.15.103:0/3170074714 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be7370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 0x5644be6c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0d3be640 1 -- 172.21.15.103:0/3170074714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 msgr2=0x5644be6c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 0x5644be6c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0d3be640 1 -- 172.21.15.103:0/3170074714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 msgr2=0x5644be7370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be7370b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.048+0000 7f2b0d3be640 1 -- 172.21.15.103:0/3170074714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644be737670 con 0x5644be6b89a0 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b0d3be640 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6c70e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2af8017010 tx=0x7f2af8016350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72fc16bdbc838b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2af17fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2af17fa640 1 -- 172.21.15.103:0/3170074714 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af802a070 con 0x5644be6b89a0 2024-09-05T02:06:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2af17fa640 1 -- 172.21.15.103:0/3170074714 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2af8003cd0 con 0x5644be6b89a0 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3170074714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b040498e0 con 0x5644be6b89a0 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2af17fa640 1 -- 172.21.15.103:0/3170074714 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2af800ce60 con 0x5644be6b89a0 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b12bbe740 10 client.25295 fetch_fsmap learned FSMap version 34 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3170074714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5644be738980 con 0x5644be6b89a0 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2af17fa640 1 -- 172.21.15.103:0/3170074714 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b040498e0 con 0x5644be6b89a0 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2af17fa640 1 -- 172.21.15.103:0/3170074714 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7f2af8016030 con 0x5644be6b89a0 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b12bbe740 10 client.25295 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b12bbe740 10 client.25295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b12bbe740 20 client.25295 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5644be739360 0x5644be73b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b12bbe740 4 client.25295 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:06:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.049+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3170074714 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644be623d70 con 0x5644be739360 2024-09-05T02:06:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.050+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5644be739360 0x5644be73b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.050+0000 7f2af17fa640 10 client.25295 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.050+0000 7f2b0dbbf640 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5644be739360 0x5644be73b780 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74e02a267e8b3b0 server_cookie=2d96d092da54358e in_seq=0 out_seq=0 2024-09-05T02:06:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.065+0000 7f2af17fa640 1 -- 172.21.15.103:0/3170074714 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644be623d70 con 0x5644be739360 2024-09-05T02:06:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.065+0000 7f2af17fa640 10 client.25295 handle_command_reply: tid=0 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.065+0000 7f2b12bbe740 1 client.25295 shutdown 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.065+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3170074714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 msgr2=0x5644be6c70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.065+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6c70e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2af8017010 tx=0x7f2af8016350 comp rx=0 tx=0).stop 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3170074714 shutdown_connections 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5644be739360 0x5644be73b780 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2af17fa640 0 client.25295 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644be6be370 0x5644be7370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644be6b92e0 0x5644be6c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2b12bbe740 1 --2- 172.21.15.103:0/3170074714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644be6b89a0 0x5644be6c70e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3170074714 >> 172.21.15.103:0/3170074714 conn(0x5644be6b4010 msgr2=0x5644be6c2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3170074714 shutdown_connections 2024-09-05T02:06:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2b12bbe740 1 -- 172.21.15.103:0/3170074714 wait complete. 2024-09-05T02:06:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.066+0000 7f2b12bbe740 20 client.25295 trim_cache size 0 max 16384 2024-09-05T02:06:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:06:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:06:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:06:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:06:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2af40a2c30 msgr2=0x7f2af40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2af40a2c30 0x7f2af40a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b00098390 tx=0x7f2b00013b60 comp rx=0 tx=0).stop 2024-09-05T02:06:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b08101d70 msgr2=0x7f2b0819d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b08101d70 0x7f2b0819d050 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009fd0 tx=0x7f2b0000ef10 comp rx=0 tx=0).stop 2024-09-05T02:06:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 shutdown_connections 2024-09-05T02:06:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b08101d70 0x7f2b0819d050 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b081013d0 0x7f2b08195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2af40a2c30 0x7f2af40a50f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 --2- 172.21.15.103:0/1337410460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b08100320 0x7f2b08195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 >> 172.21.15.103:0/1337410460 conn(0x7f2b080fb500 msgr2=0x7f2b08104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 shutdown_connections 2024-09-05T02:06:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:16.067+0000 7f2b0fe23640 1 -- 172.21.15.103:0/1337410460 wait complete. 2024-09-05T02:06:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:16 smithi167 ceph-mon[35122]: pgmap v465: 129 pgs: 129 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 592 KiB/s wr, 162 op/s 2024-09-05T02:06:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:16 smithi139 ceph-mon[34262]: pgmap v465: 129 pgs: 129 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 592 KiB/s wr, 162 op/s 2024-09-05T02:06:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:16 smithi103 ceph-mon[30902]: pgmap v465: 129 pgs: 129 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 592 KiB/s wr, 162 op/s 2024-09-05T02:06:17.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:06:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:06:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:06:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:06:17.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:06:17.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:06:17.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:06:17.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:06:17.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:06:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:06:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:06:17.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:06:17.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:06:17.780 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:06:18.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.338+0000 7fe598c4e640 1 -- 172.21.15.103:0/4211125282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 msgr2=0x7fe594103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:18.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.338+0000 7fe598c4e640 1 --2- 172.21.15.103:0/4211125282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 0x7fe594103120 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe584009900 tx=0x7fe58402a480 comp rx=0 tx=0).stop 2024-09-05T02:06:18.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.338+0000 7fe598c4e640 1 -- 172.21.15.103:0/4211125282 shutdown_connections 2024-09-05T02:06:18.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.338+0000 7fe598c4e640 1 --2- 172.21.15.103:0/4211125282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59410c170 0x7fe59410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:18.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.338+0000 7fe598c4e640 1 --2- 172.21.15.103:0/4211125282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe594103660 0x7fe59410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:18.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.338+0000 7fe598c4e640 1 --2- 172.21.15.103:0/4211125282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 0x7fe594103120 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:18.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.338+0000 7fe598c4e640 1 -- 172.21.15.103:0/4211125282 >> 172.21.15.103:0/4211125282 conn(0x7fe5940fc7a0 msgr2=0x7fe5940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:18.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.338+0000 7fe598c4e640 1 -- 172.21.15.103:0/4211125282 shutdown_connections 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.338+0000 7fe598c4e640 1 -- 172.21.15.103:0/4211125282 wait complete. 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.339+0000 7fe598c4e640 1 Processor -- start 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe598c4e640 1 -- start start 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe598c4e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 0x7fe59419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe598c4e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe594103660 0x7fe59419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe598c4e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59410c170 0x7fe59419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe598c4e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe594111e70 con 0x7fe594102d40 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe59259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 0x7fe59419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe59259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 0x7fe59419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54950/0 (socket says 172.21.15.103:54950) 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe59259c640 1 -- 172.21.15.103:0/2571829895 learned_addr learned my addr 172.21.15.103:0/2571829895 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe594111cf0 con 0x7fe594103660 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe594111ff0 con 0x7fe59410c170 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe592d9d640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59410c170 0x7fe59419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.340+0000 7fe591d9b640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe594103660 0x7fe59419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe59259c640 1 -- 172.21.15.103:0/2571829895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59410c170 msgr2=0x7fe59419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe59259c640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59410c170 0x7fe59419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe59259c640 1 -- 172.21.15.103:0/2571829895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe594103660 msgr2=0x7fe59419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe59259c640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe594103660 0x7fe59419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe59259c640 1 -- 172.21.15.103:0/2571829895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59419f530 con 0x7fe594102d40 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe591d9b640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe594103660 0x7fe59419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe59259c640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 0x7fe59419a580 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe584005ab0 tx=0x7fe584003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe5837fe640 1 -- 172.21.15.103:0/2571829895 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58403f070 con 0x7fe594102d40 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59419f7c0 con 0x7fe594102d40 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5941a7100 con 0x7fe594102d40 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe5837fe640 1 -- 172.21.15.103:0/2571829895 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe584031d60 con 0x7fe594102d40 2024-09-05T02:06:18.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.341+0000 7fe5837fe640 1 -- 172.21.15.103:0/2571829895 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584035030 con 0x7fe594102d40 2024-09-05T02:06:18.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.343+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe594103120 con 0x7fe594102d40 2024-09-05T02:06:18.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.345+0000 7fe5837fe640 1 -- 172.21.15.103:0/2571829895 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe584042070 con 0x7fe594102d40 2024-09-05T02:06:18.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.346+0000 7fe5837fe640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:18.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.346+0000 7fe5837fe640 1 -- 172.21.15.103:0/2571829895 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe5840f0260 con 0x7fe594102d40 2024-09-05T02:06:18.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.346+0000 7fe591d9b640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:18.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.347+0000 7fe591d9b640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe5940febe0 tx=0x7fe58800b690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:18.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.349+0000 7fe5837fe640 1 -- 172.21.15.103:0/2571829895 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5840bce10 con 0x7fe594102d40 2024-09-05T02:06:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:18 smithi167 ceph-mon[35122]: pgmap v466: 129 pgs: 129 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 391 KiB/s wr, 63 op/s 2024-09-05T02:06:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:18 smithi139 ceph-mon[34262]: pgmap v466: 129 pgs: 129 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 391 KiB/s wr, 63 op/s 2024-09-05T02:06:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:18 smithi103 ceph-mon[30902]: pgmap v466: 129 pgs: 129 active+clean; 219 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 391 KiB/s wr, 63 op/s 2024-09-05T02:06:18.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.701+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5941a72d0 con 0x7fe594102d40 2024-09-05T02:06:18.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.702+0000 7fe5837fe640 1 -- 172.21.15.103:0/2571829895 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+12006 (secure 0 0 0) 0x7fe5840bc710 con 0x7fe594102d40 2024-09-05T02:06:18.704 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:06:18.704 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":34,"btime":"2024-09-05T02:06:14:308540+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:06:13.693993+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:06:18.704 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 34 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe57c0a2c00 msgr2=0x7fe57c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe5940febe0 tx=0x7fe58800b690 comp rx=0 tx=0).stop 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 msgr2=0x7fe59419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 0x7fe59419a580 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe584005ab0 tx=0x7fe584003c80 comp rx=0 tx=0).stop 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 shutdown_connections 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59410c170 0x7fe59419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe594103660 0x7fe59419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 --2- 172.21.15.103:0/2571829895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe594102d40 0x7fe59419a580 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:18.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 >> 172.21.15.103:0/2571829895 conn(0x7fe5940fc7a0 msgr2=0x7fe5941008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:18.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 shutdown_connections 2024-09-05T02:06:18.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:18.705+0000 7fe598c4e640 1 -- 172.21.15.103:0/2571829895 wait complete. 2024-09-05T02:06:19.518 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id f3111ecb --release --await' 2024-09-05T02:06:19.519 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id f3111ecb --release --await 2024-09-05T02:06:19.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:19 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2571829895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:06:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:19 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2571829895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:06:19.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:19 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2571829895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:06:19.868 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:06:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.414+0000 7ff9e5798640 1 -- 172.21.15.103:0/1883293573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0102570 msgr2=0x7ff9e01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.414+0000 7ff9e5798640 1 --2- 172.21.15.103:0/1883293573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0102570 0x7ff9e01029b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009900 tx=0x7ff9d402a480 comp rx=0 tx=0).stop 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.414+0000 7ff9e5798640 1 -- 172.21.15.103:0/1883293573 shutdown_connections 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.414+0000 7ff9e5798640 1 --2- 172.21.15.103:0/1883293573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e0102ef0 0x7ff9e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.414+0000 7ff9e5798640 1 --2- 172.21.15.103:0/1883293573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0102570 0x7ff9e01029b0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.414+0000 7ff9e5798640 1 --2- 172.21.15.103:0/1883293573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9e0100e20 0x7ff9e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.414+0000 7ff9e5798640 1 -- 172.21.15.103:0/1883293573 >> 172.21.15.103:0/1883293573 conn(0x7ff9e00fc7d0 msgr2=0x7ff9e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.414+0000 7ff9e5798640 1 -- 172.21.15.103:0/1883293573 shutdown_connections 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.414+0000 7ff9e5798640 1 -- 172.21.15.103:0/1883293573 wait complete. 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.415+0000 7ff9e5798640 1 Processor -- start 2024-09-05T02:06:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.415+0000 7ff9e5798640 1 -- start start 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.415+0000 7ff9e5798640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e0100e20 0x7ff9e01a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.415+0000 7ff9e5798640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9e0102ef0 0x7ff9e01a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.415+0000 7ff9e5798640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e019a610 0x7ff9e019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.415+0000 7ff9e5798640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0110e00 con 0x7ff9e019a610 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.415+0000 7ff9e5798640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0110c80 con 0x7ff9e0100e20 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.415+0000 7ff9e5798640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0110f80 con 0x7ff9e0102ef0 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9deffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e0100e20 0x7ff9e01a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9deffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e0100e20 0x7ff9e01a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:42216/0 (socket says 172.21.15.103:42216) 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9deffd640 1 -- 172.21.15.103:0/2993447871 learned_addr learned my addr 172.21.15.103:0/2993447871 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9e0102ef0 0x7ff9e01a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e019a610 0x7ff9e019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9deffd640 1 -- 172.21.15.103:0/2993447871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9e0102ef0 msgr2=0x7ff9e01a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9deffd640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9e0102ef0 0x7ff9e01a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9deffd640 1 -- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e019a610 msgr2=0x7ff9e019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9deffd640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e019a610 0x7ff9e019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9deffd640 1 -- 172.21.15.103:0/2993447871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e019b130 con 0x7ff9e0100e20 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e019a610 0x7ff9e019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.416+0000 7ff9deffd640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e0100e20 0x7ff9e01a0450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000e9f0 tx=0x7ff9d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.417+0000 7ff9c7fff640 1 -- 172.21.15.103:0/2993447871 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000cd80 con 0x7ff9e0100e20 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.417+0000 7ff9c7fff640 1 -- 172.21.15.103:0/2993447871 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9d00108e0 con 0x7ff9e0100e20 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.417+0000 7ff9c7fff640 1 -- 172.21.15.103:0/2993447871 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0018850 con 0x7ff9e0100e20 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.417+0000 7ff9e5798640 1 -- 172.21.15.103:0/2993447871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e019b420 con 0x7ff9e0100e20 2024-09-05T02:06:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.417+0000 7ff9e5798640 1 -- 172.21.15.103:0/2993447871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e01a7610 con 0x7ff9e0100e20 2024-09-05T02:06:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9e7b71740 1 Processor -- start 2024-09-05T02:06:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9e7b71740 1 -- start start 2024-09-05T02:06:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313060 0x557d40313440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 0x557d4031dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d40320630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d40278620 con 0x557d4031e240 2024-09-05T02:06:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557d4021fe50 con 0x557d40313980 2024-09-05T02:06:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557d402745e0 con 0x557d40313060 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9de7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d40320630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313060 0x557d40313440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313060 0x557d40313440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56452/0 (socket says 172.21.15.103:56452) 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.420+0000 7ff9df7fe640 1 -- 172.21.15.103:0/2012920772 learned_addr learned my addr 172.21.15.103:0/2012920772 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.421+0000 7ff9c7fff640 1 -- 172.21.15.103:0/2993447871 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff9d0010a90 con 0x7ff9e0100e20 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.421+0000 7ff9df7fe640 1 -- 172.21.15.103:0/2012920772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 msgr2=0x557d4031dd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.421+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2012920772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 0x557d4031dd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.421+0000 7ff9df7fe640 1 -- 172.21.15.103:0/2012920772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 msgr2=0x557d40320630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.421+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2012920772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d40320630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.421+0000 7ff9df7fe640 1 -- 172.21.15.103:0/2012920772 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d40210d40 con 0x557d40313060 2024-09-05T02:06:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.421+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/2012920772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d40320630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.422+0000 7ff9c7fff640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9c80a2c30 0x7ff9c80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.422+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2012920772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313060 0x557d40313440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80117e0 tx=0x7ff9d8011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f034dcf5fc5d89a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.422+0000 7ff9c7fff640 1 -- 172.21.15.103:0/2993447871 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff9d00cf850 con 0x7ff9e0100e20 2024-09-05T02:06:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.422+0000 7ff9deffd640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9c80a2c30 0x7ff9c80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.422+0000 7ff9c57fa640 1 -- 172.21.15.103:0/2012920772 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8011310 con 0x557d40313060 2024-09-05T02:06:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.422+0000 7ff9c57fa640 1 -- 172.21.15.103:0/2012920772 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9d8003e20 con 0x557d40313060 2024-09-05T02:06:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.422+0000 7ff9c57fa640 1 -- 172.21.15.103:0/2012920772 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80044d0 con 0x557d40313060 2024-09-05T02:06:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2012920772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313060 msgr2=0x557d40313440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2012920772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313060 0x557d40313440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80117e0 tx=0x7ff9d8011ca0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9deffd640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9c80a2c30 0x7ff9c80a50f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00cfaf0 tx=0x7ff9d0009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2012920772 shutdown_connections 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2012920772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d40320630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2012920772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 0x557d4031dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2012920772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313060 0x557d40313440 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2012920772 >> 172.21.15.103:0/2012920772 conn(0x557d4030f5f0 msgr2=0x557d4030f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2012920772 shutdown_connections 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.423+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2012920772 wait complete. 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.424+0000 7ff9e7b71740 1 -- start start 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.424+0000 7ff9e7b71740 1 Processor -- start 2024-09-05T02:06:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.424+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d4031b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.424+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 0x557d4031b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.424+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d4031bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.424+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d40320e10 con 0x557d4031e240 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.424+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557d40200f10 con 0x557d40313060 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.424+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557d40320f90 con 0x557d40313980 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.424+0000 7ff9e7b71740 10 client.0 _ll_register_callbacks cb 0x557d40224db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d4031b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d4031b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:42232/0 (socket says 172.21.15.103:42232) 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4228232907 learned_addr learned my addr 172.21.15.103:0/4228232907 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d4031bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 0x557d4031b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4228232907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 msgr2=0x557d4031b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 0x557d4031b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4228232907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 msgr2=0x557d4031b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d4031b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4228232907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3ff87580 con 0x557d4031e240 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d4031bd00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800f790 tx=0x7ff9d8009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd42d57068cba59 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.425+0000 7ff9c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.426+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/4228232907 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8025040 con 0x557d4031e240 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.426+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/4228232907 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff9d8004a40 con 0x557d4031e240 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.426+0000 7ff9e7b71740 1 -- 172.21.15.103:0/4228232907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d8003e20 con 0x557d4031e240 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.426+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/4228232907 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff9d8004c40 con 0x557d4031e240 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.426+0000 7ff9e7b71740 10 client.15864 fetch_fsmap learned FSMap version 34 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.426+0000 7ff9e7b71740 1 -- 172.21.15.103:0/4228232907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557d40051960 con 0x557d4031e240 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.426+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/4228232907 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9d8003e20 con 0x557d4031e240 2024-09-05T02:06:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.426+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/4228232907 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7ff9d8018020 con 0x557d4031e240 2024-09-05T02:06:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.427+0000 7ff9e7b71740 10 client.15864 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T02:06:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.427+0000 7ff9e7b71740 10 client.15864 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:06:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.427+0000 7ff9e7b71740 20 client.15864 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.427+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557d4032e210 0x557d40330650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.427+0000 7ff9e7b71740 4 client.15864 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:06:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.427+0000 7ff9e7b71740 1 -- 172.21.15.103:0/4228232907 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d40285b90 con 0x557d4032e210 2024-09-05T02:06:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.427+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557d4032e210 0x557d40330650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557d4032e210 0x557d40330650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff5e4a99118802c server_cookie=8a4d8b0a2892a84c in_seq=0 out_seq=0 2024-09-05T02:06:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9c2ff5640 10 client.15864 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/4228232907 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d40285b90 con 0x557d4032e210 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9c2ff5640 10 client.15864 handle_command_reply: tid=0 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9e7b71740 1 client.15864 shutdown 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9e7b71740 1 -- 172.21.15.103:0/4228232907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 msgr2=0x557d4031bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d4031bd00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800f790 tx=0x7ff9d8009fd0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9e7b71740 1 -- 172.21.15.103:0/4228232907 shutdown_connections 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557d4032e210 0x557d40330650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d4031bd00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 0x557d4031b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/4228232907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d4031b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.428+0000 7ff9e7b71740 1 -- 172.21.15.103:0/4228232907 >> 172.21.15.103:0/4228232907 conn(0x557d4030f5f0 msgr2=0x557d403152d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:20.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.429+0000 7ff9c2ff5640 0 client.15864 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:20.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.429+0000 7ff9e7b71740 1 -- 172.21.15.103:0/4228232907 shutdown_connections 2024-09-05T02:06:20.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.429+0000 7ff9e7b71740 1 -- 172.21.15.103:0/4228232907 wait complete. 2024-09-05T02:06:20.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.429+0000 7ff9e7b71740 20 client.15864 trim_cache size 0 max 16384 2024-09-05T02:06:20.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.452+0000 7ff9e7b71740 1 Processor -- start 2024-09-05T02:06:20.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.452+0000 7ff9e7b71740 1 -- start start 2024-09-05T02:06:20.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.452+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d403151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.452+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 0x557d40335f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.452+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d403364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b14f0 con 0x557d4031e240 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d403151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b3120 con 0x557d40313060 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b1bc0 con 0x557d40313980 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d403151c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:42242/0 (socket says 172.21.15.103:42242) 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 -- 172.21.15.103:0/2023172475 learned_addr learned my addr 172.21.15.103:0/2023172475 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d403364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 0x557d40335f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 -- 172.21.15.103:0/2023172475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 msgr2=0x557d40335f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:20.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 0x557d40335f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 -- 172.21.15.103:0/2023172475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 msgr2=0x557d403364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d403364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 -- 172.21.15.103:0/2023172475 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d403106a0 con 0x557d40313060 2024-09-05T02:06:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 0x557d40335f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:06:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d403151c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8005920 tx=0x7ff9d8011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96bfcbcfb8d594 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9c4ff9640 1 -- 172.21.15.103:0/2023172475 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8025040 con 0x557d40313060 2024-09-05T02:06:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9c4ff9640 1 -- 172.21.15.103:0/2023172475 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9d8005290 con 0x557d40313060 2024-09-05T02:06:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.453+0000 7ff9c4ff9640 1 -- 172.21.15.103:0/2023172475 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80043a0 con 0x557d40313060 2024-09-05T02:06:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.454+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2023172475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 msgr2=0x557d403151c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.454+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d403151c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8005920 tx=0x7ff9d8011ca0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.454+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2023172475 shutdown_connections 2024-09-05T02:06:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.454+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4031e240 0x557d403364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.454+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d40313980 0x557d40335f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.454+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2023172475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313060 0x557d403151c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.454+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2023172475 >> 172.21.15.103:0/2023172475 conn(0x557d4030f5f0 msgr2=0x557d40314ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.454+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2023172475 shutdown_connections 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2023172475 wait complete. 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 1 -- start start 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 1 Processor -- start 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d40313060 0x557d403263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 0x557d40326900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4031e240 0x557d403231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b2a20 con 0x557d40313060 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b2290 con 0x557d40313980 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01af4d0 con 0x557d4031e240 2024-09-05T02:06:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9e7b71740 10 client.0 _ll_register_callbacks cb 0x7ff9d8011530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.455+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d40313060 0x557d403263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d40313060 0x557d403263c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48532/0 (socket says 172.21.15.103:48532) 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9df7fe640 1 -- 172.21.15.103:0/2399786294 learned_addr learned my addr 172.21.15.103:0/2399786294 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 0x557d40326900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4031e240 0x557d403231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9de7fc640 1 -- 172.21.15.103:0/2399786294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4031e240 msgr2=0x557d403231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4031e240 0x557d403231b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9de7fc640 1 -- 172.21.15.103:0/2399786294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d40313060 msgr2=0x557d403263c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d40313060 0x557d403263c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9de7fc640 1 -- 172.21.15.103:0/2399786294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d403238f0 con 0x557d40313980 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9de7fc640 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 0x557d40326900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4008000 tx=0x7ff9d4009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde00bea48dfe55d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/2399786294 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d404b060 con 0x557d40313980 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/2399786294 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff9d4004040 con 0x557d40313980 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.456+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2399786294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d402800a0 con 0x557d40313980 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/2399786294 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff9d4052050 con 0x557d40313980 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9e7b71740 10 client.25316 fetch_fsmap learned FSMap version 34 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2399786294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557d40399930 con 0x557d40313980 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/2399786294 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x557d402800a0 con 0x557d40313980 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/2399786294 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7ff9d4004770 con 0x557d40313980 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9e7b71740 10 client.25316 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9e7b71740 10 client.25316 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9e7b71740 20 client.25316 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557d4039a0e0 0x557d4039c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9e7b71740 4 client.25316 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f3111ecb", "release": true, "await": true}] 2024-09-05T02:06:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2399786294 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f3111ecb", "release": true, "await": true}) -- 0x557d401fc5a0 con 0x557d4039a0e0 2024-09-05T02:06:20.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.457+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557d4039a0e0 0x557d4039c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:20.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.458+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557d4039a0e0 0x557d4039c500 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1186800164621890 server_cookie=f44f0e5493332ea9 in_seq=0 out_seq=0 2024-09-05T02:06:20.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.458+0000 7ff9c2ff5640 10 client.25316 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:20.624 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:20 smithi103 ceph-mon[30902]: pgmap v467: 129 pgs: 129 active+clean; 222 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 586 KiB/s wr, 57 op/s 2024-09-05T02:06:20.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9c2ff5640 1 -- 172.21.15.103:0/2399786294 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557d401fc5a0 con 0x557d4039a0e0 2024-09-05T02:06:20.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9c2ff5640 10 client.25316 handle_command_reply: tid=0 2024-09-05T02:06:20.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 client.25316 shutdown 2024-09-05T02:06:20.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2399786294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 msgr2=0x557d40326900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:20.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 0x557d40326900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4008000 tx=0x7ff9d4009880 comp rx=0 tx=0).stop 2024-09-05T02:06:20.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2399786294 shutdown_connections 2024-09-05T02:06:20.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557d4039a0e0 0x557d4039c500 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4031e240 0x557d403231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557d40313980 0x557d40326900 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 --2- 172.21.15.103:0/2399786294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d40313060 0x557d403263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2399786294 >> 172.21.15.103:0/2399786294 conn(0x557d4030f5f0 msgr2=0x557d403152d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:20.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9c2ff5640 0 client.25316 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:20.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2399786294 shutdown_connections 2024-09-05T02:06:20.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.622+0000 7ff9e7b71740 1 -- 172.21.15.103:0/2399786294 wait complete. 2024-09-05T02:06:20.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e7b71740 20 client.25316 trim_cache size 0 max 16384 2024-09-05T02:06:20.626 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:06:20.626 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 34, 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 12, 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "f3111ecb": { 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 12, 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 141.8, 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:06:20.627 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:20.628 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:06:20.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 -- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9c80a2c30 msgr2=0x7ff9c80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:20.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9c80a2c30 0x7ff9c80a50f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00cfaf0 tx=0x7ff9d0009700 comp rx=0 tx=0).stop 2024-09-05T02:06:20.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 -- 172.21.15.103:0/2993447871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e0100e20 msgr2=0x7ff9e01a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:20.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e0100e20 0x7ff9e01a0450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000e9f0 tx=0x7ff9d000eeb0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 -- 172.21.15.103:0/2993447871 shutdown_connections 2024-09-05T02:06:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e019a610 0x7ff9e019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff9c80a2c30 0x7ff9c80a50f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9e0102ef0 0x7ff9e01a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 --2- 172.21.15.103:0/2993447871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e0100e20 0x7ff9e01a0450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.623+0000 7ff9e5798640 1 -- 172.21.15.103:0/2993447871 >> 172.21.15.103:0/2993447871 conn(0x7ff9e00fc7d0 msgr2=0x7ff9e006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.624+0000 7ff9e5798640 1 -- 172.21.15.103:0/2993447871 shutdown_connections 2024-09-05T02:06:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:20.624+0000 7ff9e5798640 1 -- 172.21.15.103:0/2993447871 wait complete. 2024-09-05T02:06:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:20 smithi167 ceph-mon[35122]: pgmap v467: 129 pgs: 129 active+clean; 222 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 586 KiB/s wr, 57 op/s 2024-09-05T02:06:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:20 smithi139 ceph-mon[34262]: pgmap v467: 129 pgs: 129 active+clean; 222 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 586 KiB/s wr, 57 op/s 2024-09-05T02:06:21.580 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f3111ecb', total seconds elapsed: 25.9 2024-09-05T02:06:21.580 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.75 2024-09-05T02:06:21.581 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:06:21.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:21 smithi167 ceph-mon[35122]: pgmap v468: 129 pgs: 129 active+clean; 224 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 636 KiB/s wr, 33 op/s 2024-09-05T02:06:21.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:21 smithi139 ceph-mon[34262]: pgmap v468: 129 pgs: 129 active+clean; 224 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 636 KiB/s wr, 33 op/s 2024-09-05T02:06:21.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:21 smithi103 ceph-mon[30902]: pgmap v468: 129 pgs: 129 active+clean; 224 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 636 KiB/s wr, 33 op/s 2024-09-05T02:06:21.941 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:06:22.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.481+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3700648826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b481027a0 msgr2=0x7f2b481067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.481+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3700648826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b481027a0 0x7f2b481067f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009900 tx=0x7f2b3c02a3f0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.481+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3700648826 shutdown_connections 2024-09-05T02:06:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.482+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3700648826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b48106f20 0x7f2b4810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.482+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3700648826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b481027a0 0x7f2b481067f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.482+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3700648826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b48101df0 0x7f2b481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.482+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3700648826 >> 172.21.15.103:0/3700648826 conn(0x7f2b480fd8f0 msgr2=0x7f2b480ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.482+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3700648826 shutdown_connections 2024-09-05T02:06:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.482+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3700648826 wait complete. 2024-09-05T02:06:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.483+0000 7f2b4e9a6640 1 Processor -- start 2024-09-05T02:06:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.483+0000 7f2b4e9a6640 1 -- start start 2024-09-05T02:06:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b4e9a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b48101df0 0x7f2b481983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b4e9a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b481027a0 0x7f2b481988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b4e9a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b48106f20 0x7f2b4819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b4e9a6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48111e30 con 0x7f2b48101df0 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b4e9a6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48111cb0 con 0x7f2b481027a0 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b4e9a6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48111fb0 con 0x7f2b48106f20 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b481027a0 0x7f2b481988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b48101df0 0x7f2b481983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b481027a0 0x7f2b481988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:42258/0 (socket says 172.21.15.103:42258) 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b48101df0 0x7f2b481983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48548/0 (socket says 172.21.15.103:48548) 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b477fe640 1 -- 172.21.15.103:0/3865830692 learned_addr learned my addr 172.21.15.103:0/3865830692 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b4cf43640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b48106f20 0x7f2b4819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b477fe640 1 -- 172.21.15.103:0/3865830692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b48106f20 msgr2=0x7f2b4819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b477fe640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b48106f20 0x7f2b4819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b477fe640 1 -- 172.21.15.103:0/3865830692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b48101df0 msgr2=0x7f2b481983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.484+0000 7f2b477fe640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b48101df0 0x7f2b481983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.485+0000 7f2b477fe640 1 -- 172.21.15.103:0/3865830692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4819d360 con 0x7f2b481027a0 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.485+0000 7f2b4cf43640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b48106f20 0x7f2b4819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.485+0000 7f2b477fe640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b481027a0 0x7f2b481988f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005940 tx=0x7f2b3c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.485+0000 7f2b457fa640 1 -- 172.21.15.103:0/3865830692 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c03f030 con 0x7f2b481027a0 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.485+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4819d5f0 con 0x7f2b481027a0 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.485+0000 7f2b457fa640 1 -- 172.21.15.103:0/3865830692 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b3c0044c0 con 0x7f2b481027a0 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.485+0000 7f2b457fa640 1 -- 172.21.15.103:0/3865830692 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c035070 con 0x7f2b481027a0 2024-09-05T02:06:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.485+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b481a4f30 con 0x7f2b481027a0 2024-09-05T02:06:22.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.487+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b481021d0 con 0x7f2b481027a0 2024-09-05T02:06:22.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.488+0000 7f2b457fa640 1 -- 172.21.15.103:0/3865830692 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2b3c042030 con 0x7f2b481027a0 2024-09-05T02:06:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.489+0000 7f2b457fa640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2b300a2c00 0x7f2b300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.489+0000 7f2b457fa640 1 -- 172.21.15.103:0/3865830692 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2b3c0ef3f0 con 0x7f2b481027a0 2024-09-05T02:06:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.489+0000 7f2b47fff640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2b300a2c00 0x7f2b300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:22.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.489+0000 7f2b47fff640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2b300a2c00 0x7f2b300a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b380095f0 tx=0x7f2b38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:22.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.493+0000 7f2b457fa640 1 -- 172.21.15.103:0/3865830692 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b3c0bbfa0 con 0x7f2b481027a0 2024-09-05T02:06:22.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.823+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b4810a160 con 0x7f2b481027a0 2024-09-05T02:06:22.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.824+0000 7f2b457fa640 1 -- 172.21.15.103:0/3865830692 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+12006 (secure 0 0 0) 0x7f2b3c0bb8a0 con 0x7f2b481027a0 2024-09-05T02:06:22.826 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:06:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":34,"btime":"2024-09-05T02:06:14:308540+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:06:13.693993+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:06:22.826 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 34 2024-09-05T02:06:22.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2b300a2c00 msgr2=0x7f2b300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:22.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2b300a2c00 0x7f2b300a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b380095f0 tx=0x7f2b38005f90 comp rx=0 tx=0).stop 2024-09-05T02:06:22.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b481027a0 msgr2=0x7f2b481988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:22.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b481027a0 0x7f2b481988f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005940 tx=0x7f2b3c0040c0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 shutdown_connections 2024-09-05T02:06:22.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2b300a2c00 0x7f2b300a50c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b48106f20 0x7f2b4819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b481027a0 0x7f2b481988f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 --2- 172.21.15.103:0/3865830692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b48101df0 0x7f2b481983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:22.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.827+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 >> 172.21.15.103:0/3865830692 conn(0x7f2b480fd8f0 msgr2=0x7f2b480feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:22.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.828+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 shutdown_connections 2024-09-05T02:06:22.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:22.828+0000 7f2b4e9a6640 1 -- 172.21.15.103:0/3865830692 wait complete. 2024-09-05T02:06:23.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:22 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3865830692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:06:23.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:22 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3865830692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:06:23.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:22 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3865830692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:06:23.684 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0' 2024-09-05T02:06:23.685 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0 2024-09-05T02:06:23.874 INFO:tasks.workunit.client.0.smithi103.stdout: 46 13668 13 8881 9 4164 29 2024-09-05T02:06:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:06:24.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:23 smithi167 ceph-mon[35122]: pgmap v469: 129 pgs: 129 active+clean; 224 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 632 KiB/s wr, 34 op/s 2024-09-05T02:06:24.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:23 smithi139 ceph-mon[34262]: pgmap v469: 129 pgs: 129 active+clean; 224 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 632 KiB/s wr, 34 op/s 2024-09-05T02:06:24.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:23 smithi103 ceph-mon[30902]: pgmap v469: 129 pgs: 129 active+clean; 224 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 632 KiB/s wr, 34 op/s 2024-09-05T02:06:24.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.592+0000 7fc162178640 1 -- 172.21.15.103:0/1595562096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c100e20 msgr2=0x7fc15c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:24.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.592+0000 7fc162178640 1 --2- 172.21.15.103:0/1595562096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c100e20 0x7fc15c101200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00fb40 tx=0x7fc14c02e640 comp rx=0 tx=0).stop 2024-09-05T02:06:24.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.592+0000 7fc162178640 1 -- 172.21.15.103:0/1595562096 shutdown_connections 2024-09-05T02:06:24.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.592+0000 7fc162178640 1 --2- 172.21.15.103:0/1595562096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc15c102ef0 0x7fc15c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.592+0000 7fc162178640 1 --2- 172.21.15.103:0/1595562096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c102570 0x7fc15c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.592+0000 7fc162178640 1 --2- 172.21.15.103:0/1595562096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c100e20 0x7fc15c101200 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.592+0000 7fc162178640 1 -- 172.21.15.103:0/1595562096 >> 172.21.15.103:0/1595562096 conn(0x7fc15c0fc7d0 msgr2=0x7fc15c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:24.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.593+0000 7fc162178640 1 -- 172.21.15.103:0/1595562096 shutdown_connections 2024-09-05T02:06:24.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.593+0000 7fc162178640 1 -- 172.21.15.103:0/1595562096 wait complete. 2024-09-05T02:06:24.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.594+0000 7fc162178640 1 Processor -- start 2024-09-05T02:06:24.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.594+0000 7fc162178640 1 -- start start 2024-09-05T02:06:24.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.594+0000 7fc162178640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c100e20 0x7fc15c1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.594+0000 7fc162178640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc15c102570 0x7fc15c196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.594+0000 7fc162178640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c102ef0 0x7fc15c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.594+0000 7fc162178640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c10fd40 con 0x7fc15c102570 2024-09-05T02:06:24.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc162178640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c10fbc0 con 0x7fc15c100e20 2024-09-05T02:06:24.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc162178640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c10fec0 con 0x7fc15c102ef0 2024-09-05T02:06:24.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c102ef0 0x7fc15c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c102ef0 0x7fc15c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56506/0 (socket says 172.21.15.103:56506) 2024-09-05T02:06:24.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15bfff640 1 -- 172.21.15.103:0/732346216 learned_addr learned my addr 172.21.15.103:0/732346216 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:24.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15affd640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc15c102570 0x7fc15c196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c100e20 0x7fc15c1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15bfff640 1 -- 172.21.15.103:0/732346216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c100e20 msgr2=0x7fc15c1962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:24.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15bfff640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c100e20 0x7fc15c1962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15bfff640 1 -- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc15c102570 msgr2=0x7fc15c196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:24.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15bfff640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc15c102570 0x7fc15c196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15bfff640 1 -- 172.21.15.103:0/732346216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c19b290 con 0x7fc15c102ef0 2024-09-05T02:06:24.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.595+0000 7fc15bfff640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c102ef0 0x7fc15c19abb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc15400eb10 tx=0x7fc15400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:24.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.596+0000 7fc158ff9640 1 -- 172.21.15.103:0/732346216 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154004b20 con 0x7fc15c102ef0 2024-09-05T02:06:24.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.596+0000 7fc162178640 1 -- 172.21.15.103:0/732346216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15c19b580 con 0x7fc15c102ef0 2024-09-05T02:06:24.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.596+0000 7fc158ff9640 1 -- 172.21.15.103:0/732346216 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc154004cc0 con 0x7fc15c102ef0 2024-09-05T02:06:24.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.596+0000 7fc158ff9640 1 -- 172.21.15.103:0/732346216 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154010690 con 0x7fc15c102ef0 2024-09-05T02:06:24.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.596+0000 7fc162178640 1 -- 172.21.15.103:0/732346216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15c1a2e60 con 0x7fc15c102ef0 2024-09-05T02:06:24.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc158ff9640 1 -- 172.21.15.103:0/732346216 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc154005020 con 0x7fc15c102ef0 2024-09-05T02:06:24.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc164551740 1 Processor -- start 2024-09-05T02:06:24.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc164551740 1 -- start start 2024-09-05T02:06:24.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc164551740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac98bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc164551740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc164551740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac996a30 0x562eac998e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc164551740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562eac8f0160 con 0x562eac98b850 2024-09-05T02:06:24.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc164551740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562eac897720 con 0x562eac98c170 2024-09-05T02:06:24.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc164551740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562eac8ec120 con 0x562eac996a30 2024-09-05T02:06:24.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc15b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc15b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:42288/0 (socket says 172.21.15.103:42288) 2024-09-05T02:06:24.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc15b7fe640 1 -- 172.21.15.103:0/1354802417 learned_addr learned my addr 172.21.15.103:0/1354802417 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc158ff9640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc1440a2c00 0x7fc1440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.599+0000 7fc15affd640 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac996a30 0x562eac998e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15affd640 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac98bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc158ff9640 1 -- 172.21.15.103:0/732346216 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc1540cf880 con 0x7fc15c102ef0 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15bfff640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc1440a2c00 0x7fc1440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15b7fe640 1 -- 172.21.15.103:0/1354802417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac996a30 msgr2=0x562eac998e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac996a30 0x562eac998e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15b7fe640 1 -- 172.21.15.103:0/1354802417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 msgr2=0x562eac98bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac98bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15b7fe640 1 -- 172.21.15.103:0/1354802417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eac661ba0 con 0x562eac98c170 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15bfff640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc1440a2c00 0x7fc1440a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1540a4470 tx=0x7fc15400efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15affd640 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac98bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:24.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.600+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9964f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00fb10 tx=0x7fc14c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7f0b6dbd92b400e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:24.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.601+0000 7fc141ffb640 1 -- 172.21.15.103:0/1354802417 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0048c0 con 0x562eac98c170 2024-09-05T02:06:24.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.601+0000 7fc141ffb640 1 -- 172.21.15.103:0/1354802417 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc14c045070 con 0x562eac98c170 2024-09-05T02:06:24.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.601+0000 7fc141ffb640 1 -- 172.21.15.103:0/1354802417 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00eb30 con 0x562eac98c170 2024-09-05T02:06:24.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.601+0000 7fc164551740 1 -- 172.21.15.103:0/1354802417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 msgr2=0x562eac9964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.601+0000 7fc164551740 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9964f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00fb10 tx=0x7fc14c005e80 comp rx=0 tx=0).stop 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.602+0000 7fc164551740 1 -- 172.21.15.103:0/1354802417 shutdown_connections 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.602+0000 7fc164551740 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac996a30 0x562eac998e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.602+0000 7fc164551740 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9964f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.602+0000 7fc164551740 1 --2- 172.21.15.103:0/1354802417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac98bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.602+0000 7fc164551740 1 -- 172.21.15.103:0/1354802417 >> 172.21.15.103:0/1354802417 conn(0x562eac987de0 msgr2=0x562eac9881c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.602+0000 7fc164551740 1 -- 172.21.15.103:0/1354802417 shutdown_connections 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.602+0000 7fc164551740 1 -- 172.21.15.103:0/1354802417 wait complete. 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.602+0000 7fc164551740 1 -- start start 2024-09-05T02:06:24.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc164551740 1 Processor -- start 2024-09-05T02:06:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc164551740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98b850 0x562eac993900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc164551740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98c170 0x562eac993e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc164551740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac996a30 0x562eac994380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc164551740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562eac999600 con 0x562eac98c170 2024-09-05T02:06:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc164551740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562eac878900 con 0x562eac996a30 2024-09-05T02:06:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc164551740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562eac999780 con 0x562eac98b850 2024-09-05T02:06:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc164551740 10 client.0 _ll_register_callbacks cb 0x562eac876780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc15affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98b850 0x562eac993900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc15affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98b850 0x562eac993900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56528/0 (socket says 172.21.15.103:56528) 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc15affd640 1 -- 172.21.15.103:0/1161957875 learned_addr learned my addr 172.21.15.103:0/1161957875 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc15affd640 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac996a30 0x562eac994380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.603+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98c170 0x562eac993e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.604+0000 7fc15affd640 1 -- 172.21.15.103:0/1161957875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac996a30 msgr2=0x562eac994380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.604+0000 7fc15affd640 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac996a30 0x562eac994380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.604+0000 7fc15affd640 1 -- 172.21.15.103:0/1161957875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98c170 msgr2=0x562eac993e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.604+0000 7fc15affd640 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98c170 0x562eac993e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.604+0000 7fc15affd640 1 -- 172.21.15.103:0/1161957875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eac8b8c50 con 0x562eac98b850 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.604+0000 7fc15affd640 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98b850 0x562eac993900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc150010f00 tx=0x7fc150009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54a15c9289051741 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:24.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.604+0000 7fc13affd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:06:24.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.605+0000 7fc13affd640 1 -- 172.21.15.103:0/1161957875 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150019040 con 0x562eac98b850 2024-09-05T02:06:24.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.605+0000 7fc13affd640 1 -- 172.21.15.103:0/1161957875 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc150005220 con 0x562eac98b850 2024-09-05T02:06:24.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.605+0000 7fc164551740 1 -- 172.21.15.103:0/1161957875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc14c045070 con 0x562eac98b850 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.605+0000 7fc13affd640 1 -- 172.21.15.103:0/1161957875 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc150005420 con 0x562eac98b850 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.605+0000 7fc164551740 10 client.25279 fetch_fsmap learned FSMap version 34 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.605+0000 7fc164551740 1 -- 172.21.15.103:0/1161957875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562eac9adaa0 con 0x562eac98b850 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.606+0000 7fc13affd640 1 -- 172.21.15.103:0/1161957875 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc14c045070 con 0x562eac98b850 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.606+0000 7fc13affd640 1 -- 172.21.15.103:0/1161957875 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7fc15002f040 con 0x562eac98b850 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.606+0000 7fc164551740 10 client.25279 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.606+0000 7fc164551740 10 client.25279 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.606+0000 7fc164551740 20 client.25279 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.606+0000 7fc164551740 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562eac9aef60 0x562eac9b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.606+0000 7fc164551740 4 client.25279 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.606+0000 7fc164551740 1 -- 172.21.15.103:0/1161957875 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eac8fd890 con 0x562eac9aef60 2024-09-05T02:06:24.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.606+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562eac9aef60 0x562eac9b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.607+0000 7fc13affd640 10 client.25279 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:24.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.607+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562eac9aef60 0x562eac9b1390 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995dd63ad9c286ea server_cookie=ef977ff64849e8a7 in_seq=0 out_seq=0 2024-09-05T02:06:24.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc13affd640 1 -- 172.21.15.103:0/1161957875 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562eac8fd890 con 0x562eac9aef60 2024-09-05T02:06:24.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc13affd640 10 client.25279 handle_command_reply: tid=0 2024-09-05T02:06:24.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc164551740 1 client.25279 shutdown 2024-09-05T02:06:24.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc164551740 1 -- 172.21.15.103:0/1161957875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98b850 msgr2=0x562eac993900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:24.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc164551740 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98b850 0x562eac993900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc150010f00 tx=0x7fc150009b50 comp rx=0 tx=0).stop 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc164551740 1 -- 172.21.15.103:0/1161957875 shutdown_connections 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc164551740 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562eac9aef60 0x562eac9b1390 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc164551740 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac996a30 0x562eac994380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc164551740 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98c170 0x562eac993e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc164551740 1 --2- 172.21.15.103:0/1161957875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98b850 0x562eac993900 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc164551740 1 -- 172.21.15.103:0/1161957875 >> 172.21.15.103:0/1161957875 conn(0x562eac987de0 msgr2=0x562eac98db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.608+0000 7fc13affd640 0 client.25279 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.609+0000 7fc164551740 1 -- 172.21.15.103:0/1161957875 shutdown_connections 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.609+0000 7fc164551740 1 -- 172.21.15.103:0/1161957875 wait complete. 2024-09-05T02:06:24.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.609+0000 7fc164551740 20 client.25279 trim_cache size 0 max 16384 2024-09-05T02:06:24.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.631+0000 7fc164551740 1 Processor -- start 2024-09-05T02:06:24.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.631+0000 7fc164551740 1 -- start start 2024-09-05T02:06:24.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.631+0000 7fc164551740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac987de0 0x562eac98e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.631+0000 7fc164551740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac9b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.631+0000 7fc164551740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.631+0000 7fc164551740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c1a42a0 con 0x562eac98b850 2024-09-05T02:06:24.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.631+0000 7fc164551740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c1a5ed0 con 0x562eac98c170 2024-09-05T02:06:24.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.631+0000 7fc164551740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c1a4970 con 0x562eac987de0 2024-09-05T02:06:24.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:42318/0 (socket says 172.21.15.103:42318) 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15affd640 1 -- 172.21.15.103:0/2587229463 learned_addr learned my addr 172.21.15.103:0/2587229463 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac9b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15affd640 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac987de0 0x562eac98e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15b7fe640 1 -- 172.21.15.103:0/2587229463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac987de0 msgr2=0x562eac98e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac987de0 0x562eac98e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15b7fe640 1 -- 172.21.15.103:0/2587229463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 msgr2=0x562eac9b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9b71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15b7fe640 1 -- 172.21.15.103:0/2587229463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eac988a70 con 0x562eac98b850 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15affd640 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac9b6c60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc14c005940 tx=0x7fc14c043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4e0665e5d67862 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.632+0000 7fc1417fa640 1 -- 172.21.15.103:0/2587229463 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0048c0 con 0x562eac98b850 2024-09-05T02:06:24.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.633+0000 7fc1417fa640 1 -- 172.21.15.103:0/2587229463 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc14c045040 con 0x562eac98b850 2024-09-05T02:06:24.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.633+0000 7fc1417fa640 1 -- 172.21.15.103:0/2587229463 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00e8d0 con 0x562eac98b850 2024-09-05T02:06:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.633+0000 7fc164551740 1 -- 172.21.15.103:0/2587229463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 msgr2=0x562eac9b6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.633+0000 7fc164551740 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac9b6c60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc14c005940 tx=0x7fc14c043040 comp rx=0 tx=0).stop 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.634+0000 7fc164551740 1 -- 172.21.15.103:0/2587229463 shutdown_connections 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.634+0000 7fc164551740 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98c170 0x562eac9b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.634+0000 7fc164551740 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac98b850 0x562eac9b6c60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.634+0000 7fc164551740 1 --2- 172.21.15.103:0/2587229463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac987de0 0x562eac98e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.634+0000 7fc164551740 1 -- 172.21.15.103:0/2587229463 >> 172.21.15.103:0/2587229463 conn(0x562eac98f2f0 msgr2=0x562eac986ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.634+0000 7fc164551740 1 -- 172.21.15.103:0/2587229463 shutdown_connections 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.634+0000 7fc164551740 1 -- 172.21.15.103:0/2587229463 wait complete. 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.634+0000 7fc164551740 1 -- start start 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.634+0000 7fc164551740 1 Processor -- start 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc164551740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac987de0 0x562eac9a3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc164551740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98b850 0x562eac9a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc164551740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98c170 0x562eaca18c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc164551740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c1a57d0 con 0x562eac987de0 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc164551740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c1a5040 con 0x562eac98b850 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc164551740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c073610 con 0x562eac98c170 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc164551740 10 client.0 _ll_register_callbacks cb 0x7fc150010d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98b850 0x562eac9a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98b850 0x562eac9a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:42332/0 (socket says 172.21.15.103:42332) 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15b7fe640 1 -- 172.21.15.103:0/3111630502 learned_addr learned my addr 172.21.15.103:0/3111630502 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15affd640 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98c170 0x562eaca18c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15affd640 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac987de0 0x562eac9a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15affd640 1 -- 172.21.15.103:0/3111630502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98b850 msgr2=0x562eac9a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15affd640 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98b850 0x562eac9a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15affd640 1 -- 172.21.15.103:0/3111630502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac987de0 msgr2=0x562eac9a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15affd640 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac987de0 0x562eac9a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.635+0000 7fc15affd640 1 -- 172.21.15.103:0/3111630502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eaca19170 con 0x562eac98c170 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.636+0000 7fc15affd640 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98c170 0x562eaca18c30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc150010010 tx=0x7fc150009bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbc79d2ead48cec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.636+0000 7fc13affd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.636+0000 7fc13affd640 1 -- 172.21.15.103:0/3111630502 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150033e20 con 0x562eac98c170 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.636+0000 7fc13affd640 1 -- 172.21.15.103:0/3111630502 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc150019040 con 0x562eac98c170 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.636+0000 7fc164551740 1 -- 172.21.15.103:0/3111630502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc14c045040 con 0x562eac98c170 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.636+0000 7fc13affd640 1 -- 172.21.15.103:0/3111630502 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc150004030 con 0x562eac98c170 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.636+0000 7fc164551740 10 client.25285 fetch_fsmap learned FSMap version 34 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.636+0000 7fc164551740 1 -- 172.21.15.103:0/3111630502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562eaca1a3d0 con 0x562eac98c170 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.636+0000 7fc13affd640 1 -- 172.21.15.103:0/3111630502 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc14c045040 con 0x562eac98c170 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc13affd640 1 -- 172.21.15.103:0/3111630502 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7fc150032690 con 0x562eac98c170 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc164551740 10 client.25285 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T02:06:24.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc164551740 10 client.25285 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:06:24.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc164551740 20 client.25285 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:24.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc164551740 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562eac996a30 0x562eaca1cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:24.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc164551740 4 client.25285 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}] 2024-09-05T02:06:24.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc164551740 1 -- 172.21.15.103:0/3111630502 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}) -- 0x562eac8f7be0 con 0x562eac996a30 2024-09-05T02:06:24.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562eac996a30 0x562eaca1cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:24.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc15b7fe640 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562eac996a30 0x562eaca1cb10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9cd7856b9fbc7de server_cookie=e64dad6cd236653b in_seq=0 out_seq=0 2024-09-05T02:06:24.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:24.637+0000 7fc13affd640 10 client.25285 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:25.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc13affd640 1 -- 172.21.15.103:0/3111630502 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562eac8f7be0 con 0x562eac996a30 2024-09-05T02:06:25.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc13affd640 10 client.25285 handle_command_reply: tid=0 2024-09-05T02:06:25.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 client.25285 shutdown 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 -- 172.21.15.103:0/3111630502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98c170 msgr2=0x562eaca18c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98c170 0x562eaca18c30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc150010010 tx=0x7fc150009bc0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 -- 172.21.15.103:0/3111630502 shutdown_connections 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x562eac996a30 0x562eaca1cb10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc13affd640 0 client.25285 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562eac98c170 0x562eaca18c30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eac98b850 0x562eac9a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 --2- 172.21.15.103:0/3111630502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562eac987de0 0x562eac9a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 -- 172.21.15.103:0/3111630502 >> 172.21.15.103:0/3111630502 conn(0x562eac98f2f0 msgr2=0x562eac995980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 -- 172.21.15.103:0/3111630502 shutdown_connections 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 1 -- 172.21.15.103:0/3111630502 wait complete. 2024-09-05T02:06:25.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.785+0000 7fc164551740 20 client.25285 trim_cache size 0 max 16384 2024-09-05T02:06:25.789 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:06:25.789 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 34, 2024-09-05T02:06:25.789 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:06:25.789 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 14, 2024-09-05T02:06:25.789 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:06:25.789 INFO:teuthology.orchestra.run.smithi103.stdout: "9aa7d699": { 2024-09-05T02:06:25.789 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 14, 2024-09-05T02:06:25.789 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 141.8, 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:25.790 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:25.791 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:25.791 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:25.791 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:25.791 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:06:25.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 -- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc1440a2c00 msgr2=0x7fc1440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:25.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc1440a2c00 0x7fc1440a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1540a4470 tx=0x7fc15400efd0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 -- 172.21.15.103:0/732346216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c102ef0 msgr2=0x7fc15c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:25.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c102ef0 0x7fc15c19abb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc15400eb10 tx=0x7fc15400c2d0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 -- 172.21.15.103:0/732346216 shutdown_connections 2024-09-05T02:06:25.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc1440a2c00 0x7fc1440a50c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c102ef0 0x7fc15c19abb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc15c102570 0x7fc15c196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 --2- 172.21.15.103:0/732346216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c100e20 0x7fc15c1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:25.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 -- 172.21.15.103:0/732346216 >> 172.21.15.103:0/732346216 conn(0x7fc15c0fc7d0 msgr2=0x7fc15c0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:25.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 -- 172.21.15.103:0/732346216 shutdown_connections 2024-09-05T02:06:25.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:25.786+0000 7fc162178640 1 -- 172.21.15.103:0/732346216 wait complete. 2024-09-05T02:06:26.593 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:26 smithi103 ceph-mon[30902]: pgmap v470: 129 pgs: 129 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 744 KiB/s wr, 50 op/s 2024-09-05T02:06:26.622 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9aa7d699', quiesce took 5.0 seconds. Will release after: 16.75 2024-09-05T02:06:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:26 smithi167 ceph-mon[35122]: pgmap v470: 129 pgs: 129 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 744 KiB/s wr, 50 op/s 2024-09-05T02:06:26.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:26 smithi139 ceph-mon[34262]: pgmap v470: 129 pgs: 129 active+clean; 227 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 744 KiB/s wr, 50 op/s 2024-09-05T02:06:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:28 smithi167 ceph-mon[35122]: pgmap v471: 129 pgs: 129 active+clean; 228 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 840 KiB/s wr, 54 op/s 2024-09-05T02:06:28.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:28 smithi139 ceph-mon[34262]: pgmap v471: 129 pgs: 129 active+clean; 228 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 840 KiB/s wr, 54 op/s 2024-09-05T02:06:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:28 smithi103 ceph-mon[30902]: pgmap v471: 129 pgs: 129 active+clean; 228 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 840 KiB/s wr, 54 op/s 2024-09-05T02:06:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.191+0000 7f1fa3a82640 1 -- 172.21.15.103:0/1056178385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f9c101c30 msgr2=0x7f1f9c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.191+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/1056178385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f9c101c30 0x7f1f9c1020b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009860 tx=0x7f1f8c02a340 comp rx=0 tx=0).stop 2024-09-05T02:06:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.191+0000 7f1fa3a82640 1 -- 172.21.15.103:0/1056178385 shutdown_connections 2024-09-05T02:06:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.191+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/1056178385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c1025f0 0x7f1f9c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.191+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/1056178385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f9c101c30 0x7f1f9c1020b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.191+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/1056178385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c100a40 0x7f1f9c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.191+0000 7f1fa3a82640 1 -- 172.21.15.103:0/1056178385 >> 172.21.15.103:0/1056178385 conn(0x7f1f9c0fc1e0 msgr2=0x7f1f9c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 -- 172.21.15.103:0/1056178385 shutdown_connections 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 -- 172.21.15.103:0/1056178385 wait complete. 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 Processor -- start 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 -- start start 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c100a40 0x7f1f9c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f9c101c30 0x7f1f9c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c1025f0 0x7f1f9c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10c550 con 0x7f1f9c100a40 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10c3d0 con 0x7f1f9c101c30 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.192+0000 7f1fa3a82640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10c6d0 con 0x7f1f9c1025f0 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa181e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c100a40 0x7f1f9c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa181e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c100a40 0x7f1f9c197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48604/0 (socket says 172.21.15.103:48604) 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa181e640 1 -- 172.21.15.103:0/2001787933 learned_addr learned my addr 172.21.15.103:0/2001787933 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa201f640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c1025f0 0x7f1f9c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa101d640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f9c101c30 0x7f1f9c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa181e640 1 -- 172.21.15.103:0/2001787933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c1025f0 msgr2=0x7f1f9c19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa181e640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c1025f0 0x7f1f9c19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa181e640 1 -- 172.21.15.103:0/2001787933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f9c101c30 msgr2=0x7f1f9c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa181e640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f9c101c30 0x7f1f9c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa181e640 1 -- 172.21.15.103:0/2001787933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c19f960 con 0x7f1f9c100a40 2024-09-05T02:06:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa101d640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f9c101c30 0x7f1f9c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:06:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1fa181e640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c100a40 0x7f1f9c197d40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800d770 tx=0x7f1f9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.193+0000 7f1f92ffd640 1 -- 172.21.15.103:0/2001787933 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98017070 con 0x7f1f9c100a40 2024-09-05T02:06:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.194+0000 7f1fa3a82640 1 -- 172.21.15.103:0/2001787933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c19fc50 con 0x7f1f9c100a40 2024-09-05T02:06:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.194+0000 7f1fa3a82640 1 -- 172.21.15.103:0/2001787933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1a01c0 con 0x7f1f9c100a40 2024-09-05T02:06:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.194+0000 7f1f92ffd640 1 -- 172.21.15.103:0/2001787933 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f98009c30 con 0x7f1f9c100a40 2024-09-05T02:06:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.194+0000 7f1f92ffd640 1 -- 172.21.15.103:0/2001787933 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98003d00 con 0x7f1f9c100a40 2024-09-05T02:06:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.197+0000 7f1fa6822740 1 Processor -- start 2024-09-05T02:06:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.197+0000 7f1fa6822740 1 -- start start 2024-09-05T02:06:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.197+0000 7f1f92ffd640 1 -- 172.21.15.103:0/2001787933 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1f98003ea0 con 0x7f1f9c100a40 2024-09-05T02:06:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.197+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8119a0 0x55ae2f811da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f816da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f817370 0x55ae2f819830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa6822740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f7752f0 con 0x55ae2f8119a0 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa6822740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f71d270 con 0x55ae2f817370 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa6822740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f7712b0 con 0x55ae2f8122e0 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa181e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f816da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa201f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8119a0 0x55ae2f811da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa201f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8119a0 0x55ae2f811da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48618/0 (socket says 172.21.15.103:48618) 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa201f640 1 -- 172.21.15.103:0/3862444154 learned_addr learned my addr 172.21.15.103:0/3862444154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa101d640 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f817370 0x55ae2f819830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa181e640 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f816da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56576/0 (socket says 172.21.15.103:56576) 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa201f640 1 -- 172.21.15.103:0/3862444154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 msgr2=0x55ae2f816da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa201f640 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f816da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa201f640 1 -- 172.21.15.103:0/3862444154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f817370 msgr2=0x55ae2f819830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa201f640 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f817370 0x55ae2f819830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa201f640 1 -- 172.21.15.103:0/3862444154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae2f71c340 con 0x55ae2f8119a0 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa181e640 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f816da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.198+0000 7f1fa201f640 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8119a0 0x55ae2f811da0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e450 tx=0x7f1f940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462926a656ad6c8c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.199+0000 7f1f92ffd640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f880a2cb0 0x7f1f880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.199+0000 7f1f87fff640 1 -- 172.21.15.103:0/3862444154 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400d910 con 0x55ae2f8119a0 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.199+0000 7f1f87fff640 1 -- 172.21.15.103:0/3862444154 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f9401c070 con 0x55ae2f8119a0 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.199+0000 7f1f87fff640 1 -- 172.21.15.103:0/3862444154 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94017e10 con 0x55ae2f8119a0 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.199+0000 7f1fa181e640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f880a2cb0 0x7f1f880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.199+0000 7f1f92ffd640 1 -- 172.21.15.103:0/2001787933 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f980cefb0 con 0x7f1f9c100a40 2024-09-05T02:06:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.200+0000 7f1fa181e640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f880a2cb0 0x7f1f880a5170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f980cca70 tx=0x7f1f980d2a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.200+0000 7f1fa6822740 1 -- 172.21.15.103:0/3862444154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8119a0 msgr2=0x55ae2f811da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.200+0000 7f1fa6822740 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8119a0 0x55ae2f811da0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e450 tx=0x7f1f940095c0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.200+0000 7f1fa6822740 1 -- 172.21.15.103:0/3862444154 shutdown_connections 2024-09-05T02:06:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.200+0000 7f1fa6822740 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f817370 0x55ae2f819830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.200+0000 7f1fa6822740 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f816da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.200+0000 7f1fa6822740 1 --2- 172.21.15.103:0/3862444154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8119a0 0x55ae2f811da0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.200+0000 7f1fa6822740 1 -- 172.21.15.103:0/3862444154 >> 172.21.15.103:0/3862444154 conn(0x55ae2f80d010 msgr2=0x55ae2f80d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.201+0000 7f1fa6822740 1 -- 172.21.15.103:0/3862444154 shutdown_connections 2024-09-05T02:06:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.201+0000 7f1fa6822740 1 -- 172.21.15.103:0/3862444154 wait complete. 2024-09-05T02:06:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.201+0000 7f1fa6822740 1 -- start start 2024-09-05T02:06:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.202+0000 7f1fa6822740 1 Processor -- start 2024-09-05T02:06:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.202+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8119a0 0x55ae2f81ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.202+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8122e0 0x55ae2f820510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.202+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f827550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.202+0000 7f1fa6822740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f81e750 con 0x55ae2f817370 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8119a0 0x55ae2f81ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8119a0 0x55ae2f81ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56578/0 (socket says 172.21.15.103:56578) 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 -- 172.21.15.103:0/2382445640 learned_addr learned my addr 172.21.15.103:0/2382445640 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa6822740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f6fe290 con 0x55ae2f8122e0 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa101d640 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8122e0 0x55ae2f820510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa6822740 1 -- 172.21.15.103:0/2382445640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f81e8d0 con 0x55ae2f8119a0 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa6822740 10 client.0 _ll_register_callbacks cb 0x55ae2f72f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f827550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 -- 172.21.15.103:0/2382445640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8122e0 msgr2=0x55ae2f820510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8122e0 0x55ae2f820510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 -- 172.21.15.103:0/2382445640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 msgr2=0x55ae2f827550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f827550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 -- 172.21.15.103:0/2382445640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae2f488210 con 0x55ae2f8119a0 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1fa201f640 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8119a0 0x55ae2f81ffd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f94017660 tx=0x7f1f94009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da162fa864e2223 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1f857fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.203+0000 7f1f857fa640 1 -- 172.21.15.103:0/2382445640 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94003970 con 0x55ae2f8119a0 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1f857fa640 1 -- 172.21.15.103:0/2382445640 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f94003b10 con 0x55ae2f8119a0 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1fa6822740 1 -- 172.21.15.103:0/2382445640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f9401c070 con 0x55ae2f8119a0 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1f857fa640 1 -- 172.21.15.103:0/2382445640 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1f94037050 con 0x55ae2f8119a0 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1fa6822740 10 client.25297 fetch_fsmap learned FSMap version 34 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1fa6822740 1 -- 172.21.15.103:0/2382445640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ae2f554620 con 0x55ae2f8119a0 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1f857fa640 1 -- 172.21.15.103:0/2382445640 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f9401c070 con 0x55ae2f8119a0 2024-09-05T02:06:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1f857fa640 1 -- 172.21.15.103:0/2382445640 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7f1f9400b040 con 0x55ae2f8119a0 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1fa6822740 10 client.25297 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1fa6822740 10 client.25297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.204+0000 7f1fa6822740 20 client.25297 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.205+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55ae2f82bae0 0x55ae2f82df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.205+0000 7f1fa6822740 4 client.25297 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.205+0000 7f1fa6822740 1 -- 172.21.15.103:0/2382445640 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae2f782a20 con 0x55ae2f82bae0 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.205+0000 7f1fa101d640 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55ae2f82bae0 0x55ae2f82df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.205+0000 7f1f857fa640 10 client.25297 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.205+0000 7f1fa101d640 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55ae2f82bae0 0x55ae2f82df60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db3037030819e130 server_cookie=92e359c0cca81701 in_seq=0 out_seq=0 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1f857fa640 1 -- 172.21.15.103:0/2382445640 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae2f782a20 con 0x55ae2f82bae0 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1f857fa640 10 client.25297 handle_command_reply: tid=0 2024-09-05T02:06:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 client.25297 shutdown 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 -- 172.21.15.103:0/2382445640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8119a0 msgr2=0x55ae2f81ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8119a0 0x55ae2f81ffd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f94017660 tx=0x7f1f94009be0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 -- 172.21.15.103:0/2382445640 shutdown_connections 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55ae2f82bae0 0x55ae2f82df60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f827550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8122e0 0x55ae2f820510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2382445640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8119a0 0x55ae2f81ffd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 -- 172.21.15.103:0/2382445640 >> 172.21.15.103:0/2382445640 conn(0x55ae2f80d010 msgr2=0x55ae2f80e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1f857fa640 0 client.25297 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 -- 172.21.15.103:0/2382445640 shutdown_connections 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.206+0000 7f1fa6822740 1 -- 172.21.15.103:0/2382445640 wait complete. 2024-09-05T02:06:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.207+0000 7f1fa6822740 20 client.25297 trim_cache size 0 max 16384 2024-09-05T02:06:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.228+0000 7f1fa6822740 1 Processor -- start 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.228+0000 7f1fa6822740 1 -- start start 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f820490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8122e0 0x55ae2f827470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f817370 0x55ae2f80df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa6822740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8000fc20 con 0x55ae2f8122e0 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa6822740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80011850 con 0x55ae2f8119a0 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa6822740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800102f0 con 0x55ae2f817370 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa201f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f820490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa101d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8122e0 0x55ae2f827470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa101d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8122e0 0x55ae2f827470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48640/0 (socket says 172.21.15.103:48640) 2024-09-05T02:06:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa201f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f820490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:42368/0 (socket says 172.21.15.103:42368) 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa101d640 1 -- 172.21.15.103:0/1960956439 learned_addr learned my addr 172.21.15.103:0/1960956439 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa201f640 1 --2- 172.21.15.103:0/1960956439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f817370 0x55ae2f80df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa101d640 1 -- 172.21.15.103:0/1960956439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f817370 msgr2=0x55ae2f80df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa101d640 1 --2- 172.21.15.103:0/1960956439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f817370 0x55ae2f80df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa101d640 1 -- 172.21.15.103:0/1960956439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 msgr2=0x55ae2f820490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa101d640 1 --2- 172.21.15.103:0/1960956439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f820490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa101d640 1 -- 172.21.15.103:0/1960956439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae2f81fb10 con 0x55ae2f8122e0 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa201f640 1 --2- 172.21.15.103:0/1960956439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f820490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.229+0000 7f1fa101d640 1 --2- 172.21.15.103:0/1960956439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8122e0 0x55ae2f827470 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c02a930 tx=0x7f1f8c030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b6172c5e80eee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.230+0000 7f1f877fe640 1 -- 172.21.15.103:0/1960956439 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c037040 con 0x55ae2f8122e0 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.230+0000 7f1f877fe640 1 -- 172.21.15.103:0/1960956439 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f8c030810 con 0x55ae2f8122e0 2024-09-05T02:06:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.230+0000 7f1f877fe640 1 -- 172.21.15.103:0/1960956439 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0046c0 con 0x55ae2f8122e0 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.230+0000 7f1fa6822740 1 -- 172.21.15.103:0/1960956439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8122e0 msgr2=0x55ae2f827470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.230+0000 7f1fa6822740 1 --2- 172.21.15.103:0/1960956439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8122e0 0x55ae2f827470 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c02a930 tx=0x7f1f8c030d50 comp rx=0 tx=0).stop 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.230+0000 7f1fa6822740 1 -- 172.21.15.103:0/1960956439 shutdown_connections 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.230+0000 7f1fa6822740 1 --2- 172.21.15.103:0/1960956439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f817370 0x55ae2f80df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.231+0000 7f1fa6822740 1 --2- 172.21.15.103:0/1960956439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f8122e0 0x55ae2f827470 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.231+0000 7f1fa6822740 1 --2- 172.21.15.103:0/1960956439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f820490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.231+0000 7f1fa6822740 1 -- 172.21.15.103:0/1960956439 >> 172.21.15.103:0/1960956439 conn(0x55ae2f80d010 msgr2=0x55ae2f81fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.231+0000 7f1fa6822740 1 -- 172.21.15.103:0/1960956439 shutdown_connections 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.231+0000 7f1fa6822740 1 -- 172.21.15.103:0/1960956439 wait complete. 2024-09-05T02:06:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.231+0000 7f1fa6822740 1 -- start start 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.231+0000 7f1fa6822740 1 Processor -- start 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.231+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f824990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f824ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa6822740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f825d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa6822740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80011150 con 0x55ae2f817370 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa6822740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800109c0 con 0x55ae2f8119a0 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa6822740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8000dc00 con 0x55ae2f8122e0 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa6822740 10 client.0 _ll_register_callbacks cb 0x7f1f8c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa201f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f825d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa101d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f824ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa201f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f825d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48646/0 (socket says 172.21.15.103:48646) 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa201f640 1 -- 172.21.15.103:0/2936443567 learned_addr learned my addr 172.21.15.103:0/2936443567 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa101d640 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f824ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56602/0 (socket says 172.21.15.103:56602) 2024-09-05T02:06:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa201f640 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f824990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa101d640 1 -- 172.21.15.103:0/2936443567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 msgr2=0x55ae2f824990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa101d640 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f824990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa101d640 1 -- 172.21.15.103:0/2936443567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 msgr2=0x55ae2f825d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa101d640 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f825d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa101d640 1 -- 172.21.15.103:0/2936443567 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae2f81eaa0 con 0x55ae2f8122e0 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa201f640 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f825d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1fa101d640 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f824ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55ae2f812a50 tx=0x7f1f8c030610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda1c0089608ed4f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.232+0000 7f1f857fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1f857fa640 1 -- 172.21.15.103:0/2936443567 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c055060 con 0x55ae2f8122e0 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1f857fa640 1 -- 172.21.15.103:0/2936443567 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f8c004040 con 0x55ae2f8122e0 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1fa6822740 1 -- 172.21.15.103:0/2936443567 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8c030810 con 0x55ae2f8122e0 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1f857fa640 1 -- 172.21.15.103:0/2936443567 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1f8c05c050 con 0x55ae2f8122e0 2024-09-05T02:06:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1fa6822740 10 client.25303 fetch_fsmap learned FSMap version 34 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1fa6822740 1 -- 172.21.15.103:0/2936443567 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ae2f8274d0 con 0x55ae2f8122e0 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1f857fa640 1 -- 172.21.15.103:0/2936443567 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f8c030810 con 0x55ae2f8122e0 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1f857fa640 1 -- 172.21.15.103:0/2936443567 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7f1f8c03d040 con 0x55ae2f8122e0 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1fa6822740 10 client.25303 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1fa6822740 10 client.25303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1fa6822740 20 client.25303 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55ae2f896af0 0x55ae2f898f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1fa6822740 4 client.25303 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.233+0000 7f1fa6822740 1 -- 172.21.15.103:0/2936443567 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae2f77cd70 con 0x55ae2f896af0 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.234+0000 7f1fa201f640 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55ae2f896af0 0x55ae2f898f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.234+0000 7f1f857fa640 10 client.25303 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.234+0000 7f1fa201f640 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55ae2f896af0 0x55ae2f898f10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513a5fe2d9f23a89 server_cookie=cd7be842cdaecfa2 in_seq=0 out_seq=0 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.263+0000 7f1f857fa640 1 -- 172.21.15.103:0/2936443567 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae2f77cd70 con 0x55ae2f896af0 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.263+0000 7f1f857fa640 10 client.25303 handle_command_reply: tid=0 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.263+0000 7f1fa6822740 1 client.25303 shutdown 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 -- 172.21.15.103:0/2936443567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 msgr2=0x55ae2f824ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f824ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55ae2f812a50 tx=0x7f1f8c030610 comp rx=0 tx=0).stop 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 -- 172.21.15.103:0/2936443567 shutdown_connections 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55ae2f896af0 0x55ae2f898f10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae2f817370 0x55ae2f825d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae2f8122e0 0x55ae2f824ed0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1f857fa640 0 client.25303 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 --2- 172.21.15.103:0/2936443567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae2f8119a0 0x55ae2f824990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 -- 172.21.15.103:0/2936443567 >> 172.21.15.103:0/2936443567 conn(0x55ae2f80d010 msgr2=0x55ae2f80b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 -- 172.21.15.103:0/2936443567 shutdown_connections 2024-09-05T02:06:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 1 -- 172.21.15.103:0/2936443567 wait complete. 2024-09-05T02:06:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.264+0000 7f1fa6822740 20 client.25303 trim_cache size 0 max 16384 2024-09-05T02:06:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:06:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:06:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:06:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:06:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 -- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f880a2cb0 msgr2=0x7f1f880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f880a2cb0 0x7f1f880a5170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f980cca70 tx=0x7f1f980d2a90 comp rx=0 tx=0).stop 2024-09-05T02:06:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 -- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c100a40 msgr2=0x7f1f9c197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c100a40 0x7f1f9c197d40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800d770 tx=0x7f1f9800dc30 comp rx=0 tx=0).stop 2024-09-05T02:06:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 -- 172.21.15.103:0/2001787933 shutdown_connections 2024-09-05T02:06:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f880a2cb0 0x7f1f880a5170 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c1025f0 0x7f1f9c19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f9c101c30 0x7f1f9c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 --2- 172.21.15.103:0/2001787933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c100a40 0x7f1f9c197d40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 -- 172.21.15.103:0/2001787933 >> 172.21.15.103:0/2001787933 conn(0x7f1f9c0fc1e0 msgr2=0x7f1f9c069990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 -- 172.21.15.103:0/2001787933 shutdown_connections 2024-09-05T02:06:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:29.265+0000 7f1fa3a82640 1 -- 172.21.15.103:0/2001787933 wait complete. 2024-09-05T02:06:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:30 smithi167 ceph-mon[35122]: pgmap v472: 129 pgs: 129 active+clean; 229 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 916 KiB/s wr, 71 op/s 2024-09-05T02:06:30.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:30 smithi139 ceph-mon[34262]: pgmap v472: 129 pgs: 129 active+clean; 229 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 916 KiB/s wr, 71 op/s 2024-09-05T02:06:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:30 smithi103 ceph-mon[30902]: pgmap v472: 129 pgs: 129 active+clean; 229 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 916 KiB/s wr, 71 op/s 2024-09-05T02:06:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:32 smithi167 ceph-mon[35122]: pgmap v473: 129 pgs: 129 active+clean; 229 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 723 KiB/s wr, 74 op/s 2024-09-05T02:06:32.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:32 smithi139 ceph-mon[34262]: pgmap v473: 129 pgs: 129 active+clean; 229 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 723 KiB/s wr, 74 op/s 2024-09-05T02:06:32.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:32 smithi103 ceph-mon[30902]: pgmap v473: 129 pgs: 129 active+clean; 229 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 723 KiB/s wr, 74 op/s 2024-09-05T02:06:33.875 INFO:tasks.workunit.client.0.smithi103.stdout: 46 11316 3 7042 5 2903 8 2024-09-05T02:06:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:34 smithi167 ceph-mon[35122]: pgmap v474: 129 pgs: 129 active+clean; 229 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 508 KiB/s wr, 75 op/s 2024-09-05T02:06:34.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:34 smithi139 ceph-mon[34262]: pgmap v474: 129 pgs: 129 active+clean; 229 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 508 KiB/s wr, 75 op/s 2024-09-05T02:06:34.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:34 smithi103 ceph-mon[30902]: pgmap v474: 129 pgs: 129 active+clean; 229 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 508 KiB/s wr, 75 op/s 2024-09-05T02:06:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:36 smithi167 ceph-mon[35122]: pgmap v475: 129 pgs: 129 active+clean; 232 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 690 KiB/s wr, 81 op/s 2024-09-05T02:06:36.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:36 smithi139 ceph-mon[34262]: pgmap v475: 129 pgs: 129 active+clean; 232 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 690 KiB/s wr, 81 op/s 2024-09-05T02:06:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:36 smithi103 ceph-mon[30902]: pgmap v475: 129 pgs: 129 active+clean; 232 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 690 KiB/s wr, 81 op/s 2024-09-05T02:06:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:38 smithi167 ceph-mon[35122]: pgmap v476: 129 pgs: 129 active+clean; 232 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 495 KiB/s wr, 63 op/s 2024-09-05T02:06:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:38 smithi139 ceph-mon[34262]: pgmap v476: 129 pgs: 129 active+clean; 232 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 495 KiB/s wr, 63 op/s 2024-09-05T02:06:38.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:38 smithi103 ceph-mon[30902]: pgmap v476: 129 pgs: 129 active+clean; 232 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 495 KiB/s wr, 63 op/s 2024-09-05T02:06:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:40 smithi167 ceph-mon[35122]: pgmap v477: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 437 KiB/s wr, 58 op/s 2024-09-05T02:06:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:40 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:06:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:40 smithi139 ceph-mon[34262]: pgmap v477: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 437 KiB/s wr, 58 op/s 2024-09-05T02:06:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:40 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:06:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:40 smithi103 ceph-mon[30902]: pgmap v477: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 437 KiB/s wr, 58 op/s 2024-09-05T02:06:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:40 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:06:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 -- 172.21.15.103:0/2906220679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54100a40 msgr2=0x7fca54100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2906220679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54100a40 0x7fca54100e40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fca50009900 tx=0x7fca5002a410 comp rx=0 tx=0).stop 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 -- 172.21.15.103:0/2906220679 shutdown_connections 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2906220679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca541025f0 0x7fca54108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2906220679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca54101c30 0x7fca541020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2906220679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54100a40 0x7fca54100e40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 -- 172.21.15.103:0/2906220679 >> 172.21.15.103:0/2906220679 conn(0x7fca540fc1e0 msgr2=0x7fca540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 -- 172.21.15.103:0/2906220679 shutdown_connections 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 -- 172.21.15.103:0/2906220679 wait complete. 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 Processor -- start 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.366+0000 7fca5b54b640 1 -- start start 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca5b54b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca54100a40 0x7fca54197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca5b54b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54101c30 0x7fca54198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca5b54b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca541025f0 0x7fca5419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca5b54b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca5410c550 con 0x7fca54101c30 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca5b54b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca5410c3d0 con 0x7fca541025f0 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca5b54b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca5410c6d0 con 0x7fca54100a40 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca58ae6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54101c30 0x7fca54198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca58ae6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54101c30 0x7fca54198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46162/0 (socket says 172.21.15.103:46162) 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca58ae6640 1 -- 172.21.15.103:0/2655204581 learned_addr learned my addr 172.21.15.103:0/2655204581 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca58ae6640 1 -- 172.21.15.103:0/2655204581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca54100a40 msgr2=0x7fca54197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca58ae6640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca54100a40 0x7fca54197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca58ae6640 1 -- 172.21.15.103:0/2655204581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca541025f0 msgr2=0x7fca5419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca58ae6640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca541025f0 0x7fca5419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca58ae6640 1 -- 172.21.15.103:0/2655204581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5419f950 con 0x7fca54101c30 2024-09-05T02:06:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca58ae6640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54101c30 0x7fca54198280 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fca4400d950 tx=0x7fca4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca4a7fc640 1 -- 172.21.15.103:0/2655204581 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44017030 con 0x7fca54101c30 2024-09-05T02:06:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca4a7fc640 1 -- 172.21.15.103:0/2655204581 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca44004020 con 0x7fca54101c30 2024-09-05T02:06:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca5b54b640 1 -- 172.21.15.103:0/2655204581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5419fc40 con 0x7fca54101c30 2024-09-05T02:06:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca4a7fc640 1 -- 172.21.15.103:0/2655204581 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca440047a0 con 0x7fca54101c30 2024-09-05T02:06:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.367+0000 7fca5b54b640 1 -- 172.21.15.103:0/2655204581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca541a0190 con 0x7fca54101c30 2024-09-05T02:06:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.370+0000 7fca4a7fc640 1 -- 172.21.15.103:0/2655204581 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fca440041d0 con 0x7fca54101c30 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.370+0000 7fca5df22740 1 Processor -- start 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.370+0000 7fca5df22740 1 -- start start 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca5df22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648fef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca5df22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ff41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca5df22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648ff6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca5df22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556648f4e2f0 con 0x556648ff4780 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca5df22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556648ef6270 con 0x556648fef6f0 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca5df22740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556648f4a2b0 con 0x556648feedb0 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca4a7fc640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca400a2c30 0x7fca400a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca58ae6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648ff6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca58ae6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648ff6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46170/0 (socket says 172.21.15.103:46170) 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca58ae6640 1 -- 172.21.15.103:0/3852757229 learned_addr learned my addr 172.21.15.103:0/3852757229 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca592e7640 1 --2- 172.21.15.103:0/3852757229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ff41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca59ae8640 1 --2- 172.21.15.103:0/3852757229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648fef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca59ae8640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca400a2c30 0x7fca400a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca4a7fc640 1 -- 172.21.15.103:0/2655204581 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fca440cf040 con 0x7fca54101c30 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca592e7640 1 -- 172.21.15.103:0/3852757229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 msgr2=0x556648fef1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca592e7640 1 --2- 172.21.15.103:0/3852757229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648fef1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca592e7640 1 -- 172.21.15.103:0/3852757229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 msgr2=0x556648ff6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca592e7640 1 --2- 172.21.15.103:0/3852757229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648ff6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca592e7640 1 -- 172.21.15.103:0/3852757229 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556648ef5340 con 0x556648fef6f0 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.371+0000 7fca59ae8640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca400a2c30 0x7fca400a50f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fca4c005fd0 tx=0x7fca4c0094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.372+0000 7fca592e7640 1 --2- 172.21.15.103:0/3852757229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ff41b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fca5002a930 tx=0x7fca50000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c56c036056f674be server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.372+0000 7fca3f7fe640 1 -- 172.21.15.103:0/3852757229 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50040020 con 0x556648fef6f0 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.372+0000 7fca3f7fe640 1 -- 172.21.15.103:0/3852757229 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca50004040 con 0x556648fef6f0 2024-09-05T02:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.372+0000 7fca3f7fe640 1 -- 172.21.15.103:0/3852757229 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50035040 con 0x556648fef6f0 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.372+0000 7fca5df22740 1 -- 172.21.15.103:0/3852757229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 msgr2=0x556648ff41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.372+0000 7fca5df22740 1 --2- 172.21.15.103:0/3852757229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ff41b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fca5002a930 tx=0x7fca50000bc0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 -- 172.21.15.103:0/3852757229 shutdown_connections 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 --2- 172.21.15.103:0/3852757229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648ff6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 --2- 172.21.15.103:0/3852757229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ff41b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 --2- 172.21.15.103:0/3852757229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648fef1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 -- 172.21.15.103:0/3852757229 >> 172.21.15.103:0/3852757229 conn(0x556648fea420 msgr2=0x556648fea820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 -- 172.21.15.103:0/3852757229 shutdown_connections 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 -- 172.21.15.103:0/3852757229 wait complete. 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 -- start start 2024-09-05T02:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 Processor -- start 2024-09-05T02:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648ff9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ff95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556649000530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556648ff76f0 con 0x556648ff4780 2024-09-05T02:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556648ed7290 con 0x556648fef6f0 2024-09-05T02:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556648ff7870 con 0x556648feedb0 2024-09-05T02:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.373+0000 7fca5df22740 10 client.0 _ll_register_callbacks cb 0x556648f08140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca59ae8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556649000530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca59ae8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556649000530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46186/0 (socket says 172.21.15.103:46186) 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca59ae8640 1 -- 172.21.15.103:0/50913799 learned_addr learned my addr 172.21.15.103:0/50913799 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca58ae6640 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ff95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca592e7640 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648ff9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca59ae8640 1 -- 172.21.15.103:0/50913799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 msgr2=0x556648ff9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca59ae8640 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648ff9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca59ae8640 1 -- 172.21.15.103:0/50913799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 msgr2=0x556648ff95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca59ae8640 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ff95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca59ae8640 1 -- 172.21.15.103:0/50913799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556648c61210 con 0x556648ff4780 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca59ae8640 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556649000530 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fca4c003ae0 tx=0x7fca4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4820ae6c12b4ffcd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca3cff9640 1 -- 172.21.15.103:0/50913799 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c023030 con 0x556648ff4780 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca3cff9640 1 -- 172.21.15.103:0/50913799 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fca4c004240 con 0x556648ff4780 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca5df22740 1 -- 172.21.15.103:0/50913799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca50004040 con 0x556648ff4780 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca3cff9640 1 -- 172.21.15.103:0/50913799 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fca4c030050 con 0x556648ff4780 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca5df22740 10 client.15924 fetch_fsmap learned FSMap version 35 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.374+0000 7fca5df22740 1 -- 172.21.15.103:0/50913799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556648d2d620 con 0x556648ff4780 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.375+0000 7fca3cff9640 1 -- 172.21.15.103:0/50913799 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca50004040 con 0x556648ff4780 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.375+0000 7fca3cff9640 1 -- 172.21.15.103:0/50913799 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fca4c01fb70 con 0x556648ff4780 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.375+0000 7fca5df22740 10 client.15924 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.375+0000 7fca5df22740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.375+0000 7fca5df22740 20 client.15924 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.375+0000 7fca5df22740 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556649004bd0 0x556649007000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.375+0000 7fca5df22740 4 client.15924 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.375+0000 7fca5df22740 1 -- 172.21.15.103:0/50913799 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556648f5ba20 con 0x556649004bd0 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.375+0000 7fca592e7640 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556649004bd0 0x556649007000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca592e7640 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556649004bd0 0x556649007000 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d360a2bf8d4ca84 server_cookie=14d40fdf896c4c1f in_seq=0 out_seq=0 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca3cff9640 10 client.15924 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca3cff9640 1 -- 172.21.15.103:0/50913799 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556648f5ba20 con 0x556649004bd0 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca3cff9640 10 client.15924 handle_command_reply: tid=0 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca5df22740 1 client.15924 shutdown 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca5df22740 1 -- 172.21.15.103:0/50913799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 msgr2=0x556649000530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca5df22740 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556649000530 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fca4c003ae0 tx=0x7fca4c004a40 comp rx=0 tx=0).stop 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca5df22740 1 -- 172.21.15.103:0/50913799 shutdown_connections 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca5df22740 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556649004bd0 0x556649007000 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.376+0000 7fca5df22740 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556649000530 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.377+0000 7fca5df22740 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ff95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.377+0000 7fca3cff9640 0 client.15924 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.377+0000 7fca5df22740 1 --2- 172.21.15.103:0/50913799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648ff9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.377+0000 7fca5df22740 1 -- 172.21.15.103:0/50913799 >> 172.21.15.103:0/50913799 conn(0x556648fea420 msgr2=0x556648fe51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.377+0000 7fca5df22740 1 -- 172.21.15.103:0/50913799 shutdown_connections 2024-09-05T02:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.377+0000 7fca5df22740 1 -- 172.21.15.103:0/50913799 wait complete. 2024-09-05T02:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.377+0000 7fca5df22740 20 client.15924 trim_cache size 0 max 16384 2024-09-05T02:06:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca5df22740 1 Processor -- start 2024-09-05T02:06:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca5df22740 1 -- start start 2024-09-05T02:06:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca5df22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648feedb0 0x556648feb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca5df22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648fef6f0 0x5566490004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca5df22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648ff4780 0x556648feb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca5df22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca541a8470 con 0x556648feedb0 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca5df22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca541aa070 con 0x556648ff4780 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca5df22740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca541a8b40 con 0x556648fef6f0 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca592e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648feedb0 0x556648feb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca592e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648feedb0 0x556648feb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46194/0 (socket says 172.21.15.103:46194) 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.391+0000 7fca592e7640 1 -- 172.21.15.103:0/1497054370 learned_addr learned my addr 172.21.15.103:0/1497054370 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca592e7640 1 -- 172.21.15.103:0/1497054370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648fef6f0 msgr2=0x5566490004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca59ae8640 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648ff4780 0x556648feb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca58ae6640 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648fef6f0 0x5566490004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca592e7640 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648fef6f0 0x5566490004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca592e7640 1 -- 172.21.15.103:0/1497054370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648ff4780 msgr2=0x556648feb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca592e7640 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648ff4780 0x556648feb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca592e7640 1 -- 172.21.15.103:0/1497054370 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556648fefbd0 con 0x556648feedb0 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca58ae6640 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648fef6f0 0x5566490004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca59ae8640 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648ff4780 0x556648feb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca592e7640 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648feedb0 0x556648feb2d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fca5002a930 tx=0x7fca50032b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdf14a4200fa883 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca3effd640 1 -- 172.21.15.103:0/1497054370 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50040070 con 0x556648feedb0 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca3effd640 1 -- 172.21.15.103:0/1497054370 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca5003e040 con 0x556648feedb0 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.392+0000 7fca3effd640 1 -- 172.21.15.103:0/1497054370 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50035040 con 0x556648feedb0 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 -- 172.21.15.103:0/1497054370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648feedb0 msgr2=0x556648feb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648feedb0 0x556648feb2d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fca5002a930 tx=0x7fca50032b80 comp rx=0 tx=0).stop 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 -- 172.21.15.103:0/1497054370 shutdown_connections 2024-09-05T02:06:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648ff4780 0x556648feb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648fef6f0 0x5566490004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 --2- 172.21.15.103:0/1497054370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648feedb0 0x556648feb2d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 -- 172.21.15.103:0/1497054370 >> 172.21.15.103:0/1497054370 conn(0x556648fea420 msgr2=0x556648ff9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 -- 172.21.15.103:0/1497054370 shutdown_connections 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 -- 172.21.15.103:0/1497054370 wait complete. 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 -- start start 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 Processor -- start 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648ffde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ffe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648fff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca541a99a0 con 0x556648ff4780 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca541a9210 con 0x556648fef6f0 2024-09-05T02:06:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca541a6450 con 0x556648feedb0 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca5df22740 10 client.0 _ll_register_callbacks cb 0x7fca4c003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.393+0000 7fca59ae8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648fff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca59ae8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648fff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46210/0 (socket says 172.21.15.103:46210) 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca59ae8640 1 -- 172.21.15.103:0/67171720 learned_addr learned my addr 172.21.15.103:0/67171720 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca58ae6640 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ffe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca592e7640 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648ffde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca59ae8640 1 -- 172.21.15.103:0/67171720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 msgr2=0x556648ffde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca59ae8640 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648ffde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca59ae8640 1 -- 172.21.15.103:0/67171720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 msgr2=0x556648ffe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca59ae8640 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ffe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca59ae8640 1 -- 172.21.15.103:0/67171720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556648fff9a0 con 0x556648ff4780 2024-09-05T02:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca58ae6640 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ffe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca59ae8640 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648fff390 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fca4c014010 tx=0x7fca4c00ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5870fd89e2684935 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca3cff9640 1 -- 172.21.15.103:0/67171720 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c038020 con 0x556648ff4780 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca3cff9640 1 -- 172.21.15.103:0/67171720 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fca4c004b20 con 0x556648ff4780 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca5df22740 1 -- 172.21.15.103:0/67171720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5003e040 con 0x556648ff4780 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca3cff9640 1 -- 172.21.15.103:0/67171720 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fca4c023070 con 0x556648ff4780 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca5df22740 10 client.15930 fetch_fsmap learned FSMap version 35 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.394+0000 7fca5df22740 1 -- 172.21.15.103:0/67171720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55664906fa00 con 0x556648ff4780 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.395+0000 7fca3cff9640 1 -- 172.21.15.103:0/67171720 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca5003e040 con 0x556648ff4780 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.395+0000 7fca3cff9640 1 -- 172.21.15.103:0/67171720 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fca4c004240 con 0x556648ff4780 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.395+0000 7fca5df22740 10 client.15930 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:06:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.395+0000 7fca5df22740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.395+0000 7fca5df22740 20 client.15930 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.395+0000 7fca5df22740 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556649070350 0x556649072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.395+0000 7fca5df22740 4 client.15930 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:06:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.395+0000 7fca5df22740 1 -- 172.21.15.103:0/67171720 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556648f55d70 con 0x556649070350 2024-09-05T02:06:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.395+0000 7fca592e7640 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556649070350 0x556649072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.396+0000 7fca592e7640 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556649070350 0x556649072770 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33201bf935997a5d server_cookie=cb48e4108fc9bf47 in_seq=0 out_seq=0 2024-09-05T02:06:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.396+0000 7fca3cff9640 10 client.15930 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca3cff9640 1 -- 172.21.15.103:0/67171720 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556648f55d70 con 0x556649070350 2024-09-05T02:06:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca3cff9640 10 client.15930 handle_command_reply: tid=0 2024-09-05T02:06:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 client.15930 shutdown 2024-09-05T02:06:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 -- 172.21.15.103:0/67171720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 msgr2=0x556648fff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648fff390 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fca4c014010 tx=0x7fca4c00ff80 comp rx=0 tx=0).stop 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 -- 172.21.15.103:0/67171720 shutdown_connections 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x556649070350 0x556649072770 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556648ff4780 0x556648fff390 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556648fef6f0 0x556648ffe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca3cff9640 0 client.15930 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 --2- 172.21.15.103:0/67171720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556648feedb0 0x556648ffde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 -- 172.21.15.103:0/67171720 >> 172.21.15.103:0/67171720 conn(0x556648fea420 msgr2=0x556648fe49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 -- 172.21.15.103:0/67171720 shutdown_connections 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 1 -- 172.21.15.103:0/67171720 wait complete. 2024-09-05T02:06:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.405+0000 7fca5df22740 20 client.15930 trim_cache size 0 max 16384 2024-09-05T02:06:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:06:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:06:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:06:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 -- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca400a2c30 msgr2=0x7fca400a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca400a2c30 0x7fca400a50f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fca4c005fd0 tx=0x7fca4c0094b0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 -- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54101c30 msgr2=0x7fca54198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54101c30 0x7fca54198280 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fca4400d950 tx=0x7fca4400de10 comp rx=0 tx=0).stop 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 -- 172.21.15.103:0/2655204581 shutdown_connections 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca541025f0 0x7fca5419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54101c30 0x7fca54198280 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fca400a2c30 0x7fca400a50f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 --2- 172.21.15.103:0/2655204581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca54100a40 0x7fca54197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 -- 172.21.15.103:0/2655204581 >> 172.21.15.103:0/2655204581 conn(0x7fca540fc1e0 msgr2=0x7fca540698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 -- 172.21.15.103:0/2655204581 shutdown_connections 2024-09-05T02:06:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:42.406+0000 7fca5b54b640 1 -- 172.21.15.103:0/2655204581 wait complete. 2024-09-05T02:06:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:42 smithi167 ceph-mon[35122]: pgmap v478: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 306 KiB/s wr, 38 op/s 2024-09-05T02:06:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:42 smithi139 ceph-mon[34262]: pgmap v478: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 306 KiB/s wr, 38 op/s 2024-09-05T02:06:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:42 smithi103 ceph-mon[30902]: pgmap v478: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 306 KiB/s wr, 38 op/s 2024-09-05T02:06:43.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:06:43.636 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:06:43.875 INFO:tasks.workunit.client.0.smithi103.stdout: 46 409 0 266 0 72 0 2024-09-05T02:06:44.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:44 smithi103 ceph-mon[30902]: pgmap v479: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 302 KiB/s wr, 26 op/s 2024-09-05T02:06:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:44 smithi167 ceph-mon[35122]: pgmap v479: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 302 KiB/s wr, 26 op/s 2024-09-05T02:06:44.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:44 smithi139 ceph-mon[34262]: pgmap v479: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 302 KiB/s wr, 26 op/s 2024-09-05T02:06:45.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 -- 172.21.15.103:0/3992097885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc102780 msgr2=0x7f60bc1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 --2- 172.21.15.103:0/3992097885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc102780 0x7f60bc1067d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009900 tx=0x7f60a802a410 comp rx=0 tx=0).stop 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 -- 172.21.15.103:0/3992097885 shutdown_connections 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 --2- 172.21.15.103:0/3992097885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60bc106f00 0x7f60bc10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 --2- 172.21.15.103:0/3992097885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc102780 0x7f60bc1067d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 --2- 172.21.15.103:0/3992097885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60bc101dd0 0x7f60bc1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 -- 172.21.15.103:0/3992097885 >> 172.21.15.103:0/3992097885 conn(0x7f60bc0fd8b0 msgr2=0x7f60bc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 -- 172.21.15.103:0/3992097885 shutdown_connections 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 -- 172.21.15.103:0/3992097885 wait complete. 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 Processor -- start 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.711+0000 7f60c30ed640 1 -- start start 2024-09-05T02:06:45.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c30ed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc101dd0 0x7f60bc198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c30ed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60bc102780 0x7f60bc198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c30ed640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60bc106f00 0x7f60bc19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c30ed640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc111e30 con 0x7f60bc101dd0 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c30ed640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc111cb0 con 0x7f60bc106f00 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c30ed640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc111fb0 con 0x7f60bc102780 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c0e89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc101dd0 0x7f60bc198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c0e89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc101dd0 0x7f60bc198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46230/0 (socket says 172.21.15.103:46230) 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c0e89640 1 -- 172.21.15.103:0/778934714 learned_addr learned my addr 172.21.15.103:0/778934714 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c168a640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60bc106f00 0x7f60bc19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60bbfff640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60bc102780 0x7f60bc198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c0e89640 1 -- 172.21.15.103:0/778934714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60bc102780 msgr2=0x7f60bc198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c0e89640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60bc102780 0x7f60bc198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c0e89640 1 -- 172.21.15.103:0/778934714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60bc106f00 msgr2=0x7f60bc19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c0e89640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60bc106f00 0x7f60bc19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c0e89640 1 -- 172.21.15.103:0/778934714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc19d4e0 con 0x7f60bc101dd0 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.712+0000 7f60c0e89640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc101dd0 0x7f60bc198530 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f60b000b4a0 tx=0x7f60b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.713+0000 7f60b9ffb640 1 -- 172.21.15.103:0/778934714 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b001d070 con 0x7f60bc101dd0 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.713+0000 7f60b9ffb640 1 -- 172.21.15.103:0/778934714 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60b0018440 con 0x7f60bc101dd0 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.713+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc19d7d0 con 0x7f60bc101dd0 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.713+0000 7f60b9ffb640 1 -- 172.21.15.103:0/778934714 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0016580 con 0x7f60bc101dd0 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.713+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc074420 con 0x7f60bc101dd0 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.714+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc1021b0 con 0x7f60bc101dd0 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.715+0000 7f60b9ffb640 1 -- 172.21.15.103:0/778934714 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f60b0020070 con 0x7f60bc101dd0 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.715+0000 7f60b9ffb640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60ac0a2c00 0x7f60ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:45.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.716+0000 7f60bbfff640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60ac0a2c00 0x7f60ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:45.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.716+0000 7f60b9ffb640 1 -- 172.21.15.103:0/778934714 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f60b00d25d0 con 0x7f60bc101dd0 2024-09-05T02:06:45.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.716+0000 7f60bbfff640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60ac0a2c00 0x7f60ac0a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f60a8031440 tx=0x7f60a80095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:45.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.720+0000 7f60b9ffb640 1 -- 172.21.15.103:0/778934714 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60b009f180 con 0x7f60bc101dd0 2024-09-05T02:06:45.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.922+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60bc10a0e0 con 0x7f60bc101dd0 2024-09-05T02:06:45.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.923+0000 7f60b9ffb640 1 -- 172.21.15.103:0/778934714 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+12005 (secure 0 0 0) 0x7f60b009ea80 con 0x7f60bc101dd0 2024-09-05T02:06:45.924 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:06:45.924 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":35,"btime":"2024-09-05T02:06:39:264756+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:06:38.694889+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:06:45.924 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 35 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60ac0a2c00 msgr2=0x7f60ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60ac0a2c00 0x7f60ac0a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f60a8031440 tx=0x7f60a80095f0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc101dd0 msgr2=0x7f60bc198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc101dd0 0x7f60bc198530 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f60b000b4a0 tx=0x7f60b000b960 comp rx=0 tx=0).stop 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 shutdown_connections 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60bc106f00 0x7f60bc19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f60ac0a2c00 0x7f60ac0a50c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60bc102780 0x7f60bc198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 --2- 172.21.15.103:0/778934714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60bc101dd0 0x7f60bc198530 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 >> 172.21.15.103:0/778934714 conn(0x7f60bc0fd8b0 msgr2=0x7f60bc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:45.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 shutdown_connections 2024-09-05T02:06:45.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:45.925+0000 7f60c30ed640 1 -- 172.21.15.103:0/778934714 wait complete. 2024-09-05T02:06:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:46 smithi167 ceph-mon[35122]: pgmap v480: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 304 KiB/s wr, 24 op/s 2024-09-05T02:06:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:46 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/778934714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:06:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:46 smithi139 ceph-mon[34262]: pgmap v480: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 304 KiB/s wr, 24 op/s 2024-09-05T02:06:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:46 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/778934714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:06:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:46 smithi103 ceph-mon[30902]: pgmap v480: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 304 KiB/s wr, 24 op/s 2024-09-05T02:06:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:46 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/778934714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:06:46.747 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 9aa7d699 --release --await' 2024-09-05T02:06:46.748 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 9aa7d699 --release --await 2024-09-05T02:06:47.005 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:06:47.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.494+0000 7f0d6987f640 1 -- 172.21.15.103:0/763388094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64101420 msgr2=0x7f0d6410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:47.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.494+0000 7f0d6987f640 1 --2- 172.21.15.103:0/763388094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64101420 0x7f0d6410d950 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009cd0 tx=0x7f0d5c02a8e0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.494+0000 7f0d6987f640 1 -- 172.21.15.103:0/763388094 shutdown_connections 2024-09-05T02:06:47.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.494+0000 7f0d6987f640 1 --2- 172.21.15.103:0/763388094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64101420 0x7f0d6410d950 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.494+0000 7f0d6987f640 1 --2- 172.21.15.103:0/763388094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d64100a80 0x7f0d64100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.494+0000 7f0d6987f640 1 --2- 172.21.15.103:0/763388094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d64106a80 0x7f0d64106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.494+0000 7f0d6987f640 1 -- 172.21.15.103:0/763388094 >> 172.21.15.103:0/763388094 conn(0x7f0d640fc7a0 msgr2=0x7f0d640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.494+0000 7f0d6987f640 1 -- 172.21.15.103:0/763388094 shutdown_connections 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.494+0000 7f0d6987f640 1 -- 172.21.15.103:0/763388094 wait complete. 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d6987f640 1 Processor -- start 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d6987f640 1 -- start start 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d6987f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d64100a80 0x7f0d64197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d6987f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d64101420 0x7f0d64197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d6987f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64106a80 0x7f0d6419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d6987f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d64110d30 con 0x7f0d64106a80 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d6987f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d64110bb0 con 0x7f0d64100a80 2024-09-05T02:06:47.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d6987f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d64110eb0 con 0x7f0d64101420 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64106a80 0x7f0d6419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64106a80 0x7f0d6419bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46242/0 (socket says 172.21.15.103:46242) 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.495+0000 7f0d637fe640 1 -- 172.21.15.103:0/395890101 learned_addr learned my addr 172.21.15.103:0/395890101 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d64100a80 0x7f0d64197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d627fc640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d64101420 0x7f0d64197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d637fe640 1 -- 172.21.15.103:0/395890101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d64101420 msgr2=0x7f0d64197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d637fe640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d64101420 0x7f0d64197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d637fe640 1 -- 172.21.15.103:0/395890101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d64100a80 msgr2=0x7f0d64197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d637fe640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d64100a80 0x7f0d64197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d637fe640 1 -- 172.21.15.103:0/395890101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6419c1e0 con 0x7f0d64106a80 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d64100a80 0x7f0d64197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:06:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d637fe640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64106a80 0x7f0d6419bb00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c005cc0 tx=0x7f0d5c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d6887d640 1 -- 172.21.15.103:0/395890101 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c004450 con 0x7f0d64106a80 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d6987f640 1 -- 172.21.15.103:0/395890101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6419c470 con 0x7f0d64106a80 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d6887d640 1 -- 172.21.15.103:0/395890101 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d5c031c90 con 0x7f0d64106a80 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d6887d640 1 -- 172.21.15.103:0/395890101 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c03b400 con 0x7f0d64106a80 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.496+0000 7f0d6987f640 1 -- 172.21.15.103:0/395890101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d641a3db0 con 0x7f0d64106a80 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.498+0000 7f0d6887d640 1 -- 172.21.15.103:0/395890101 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0d5c004680 con 0x7f0d64106a80 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d6bc58740 1 Processor -- start 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d6bc58740 1 -- start start 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd076f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd079840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557cdcfcf620 con 0x557cdd06a7c0 2024-09-05T02:06:47.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557cdcf76e50 con 0x557cdd069ea0 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557cdcfcb5e0 con 0x557cdd077450 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd076f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46244/0 (socket says 172.21.15.103:46244) 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d62ffd640 1 -- 172.21.15.103:0/802731806 learned_addr learned my addr 172.21.15.103:0/802731806 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d627fc640 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d6887d640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d500a2b80 0x7f0d500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d627fc640 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd079840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d62ffd640 1 -- 172.21.15.103:0/802731806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 msgr2=0x557cdd079840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd079840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d637fe640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d500a2b80 0x7f0d500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d62ffd640 1 -- 172.21.15.103:0/802731806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 msgr2=0x557cdd06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d62ffd640 1 -- 172.21.15.103:0/802731806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cdcf67d40 con 0x557cdd06a7c0 2024-09-05T02:06:47.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.499+0000 7f0d627fc640 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.500+0000 7f0d6887d640 1 -- 172.21.15.103:0/395890101 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0d5c02e030 con 0x7f0d64106a80 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.500+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd076f10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009bc0 tx=0x7f0d5400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944e55baf485dcb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.500+0000 7f0d59ffb640 1 -- 172.21.15.103:0/802731806 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54016070 con 0x557cdd06a7c0 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.500+0000 7f0d637fe640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d500a2b80 0x7f0d500a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0b8d60 tx=0x7f0d5c08e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.500+0000 7f0d59ffb640 1 -- 172.21.15.103:0/802731806 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d540054d0 con 0x557cdd06a7c0 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.500+0000 7f0d59ffb640 1 -- 172.21.15.103:0/802731806 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54011470 con 0x557cdd06a7c0 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 -- 172.21.15.103:0/802731806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 msgr2=0x557cdd076f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd076f10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009bc0 tx=0x7f0d5400e6c0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 -- 172.21.15.103:0/802731806 shutdown_connections 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd079840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd076f10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/802731806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 -- 172.21.15.103:0/802731806 >> 172.21.15.103:0/802731806 conn(0x557cdd0665f0 msgr2=0x557cdd0669d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 -- 172.21.15.103:0/802731806 shutdown_connections 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 -- 172.21.15.103:0/802731806 wait complete. 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 -- start start 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.501+0000 7f0d6bc58740 1 Processor -- start 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd0743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd0748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd074e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557cdd07a020 con 0x557cdd06a7c0 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557cdcf57f10 con 0x557cdd069ea0 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557cdd07a1a0 con 0x557cdd077450 2024-09-05T02:06:47.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d6bc58740 10 client.0 _ll_register_callbacks cb 0x557cdcf7bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd0748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd0748e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46250/0 (socket says 172.21.15.103:46250) 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d62ffd640 1 -- 172.21.15.103:0/2010325379 learned_addr learned my addr 172.21.15.103:0/2010325379 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d627fc640 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd0743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d62ffd640 1 -- 172.21.15.103:0/2010325379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 msgr2=0x557cdd074e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d627fc640 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd074e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd074e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d62ffd640 1 -- 172.21.15.103:0/2010325379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 msgr2=0x557cdd0743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd0743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d62ffd640 1 -- 172.21.15.103:0/2010325379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cdccde580 con 0x557cdd06a7c0 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d627fc640 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd0748e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0d54004f30 tx=0x7f0d54004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2749ba541819b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:47.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d42ffd640 1 -- 172.21.15.103:0/2010325379 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54016040 con 0x557cdd06a7c0 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d6bc58740 1 -- 172.21.15.103:0/2010325379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d540054d0 con 0x557cdd06a7c0 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.502+0000 7f0d42ffd640 1 -- 172.21.15.103:0/2010325379 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0d54005210 con 0x557cdd06a7c0 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d42ffd640 1 -- 172.21.15.103:0/2010325379 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0d5402e050 con 0x557cdd06a7c0 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d6bc58740 10 client.15948 fetch_fsmap learned FSMap version 35 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d6bc58740 1 -- 172.21.15.103:0/2010325379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557cdcda8960 con 0x557cdd06a7c0 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d42ffd640 1 -- 172.21.15.103:0/2010325379 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d540054d0 con 0x557cdd06a7c0 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d42ffd640 1 -- 172.21.15.103:0/2010325379 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f0d5400f040 con 0x557cdd06a7c0 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d6bc58740 10 client.15948 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d6bc58740 10 client.15948 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d6bc58740 20 client.15948 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557cdd086f50 0x557cdd089340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d6bc58740 4 client.15948 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d6bc58740 1 -- 172.21.15.103:0/2010325379 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cdcfdcb90 con 0x557cdd086f50 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.503+0000 7f0d627fc640 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557cdd086f50 0x557cdd089340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.504+0000 7f0d627fc640 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557cdd086f50 0x557cdd089340 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ba206506626aba server_cookie=3f00a8018c2a5a81 in_seq=0 out_seq=0 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.504+0000 7f0d42ffd640 10 client.15948 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.504+0000 7f0d42ffd640 1 -- 172.21.15.103:0/2010325379 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cdcfdcb90 con 0x557cdd086f50 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.504+0000 7f0d42ffd640 10 client.15948 handle_command_reply: tid=0 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.504+0000 7f0d6bc58740 1 client.15948 shutdown 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 -- 172.21.15.103:0/2010325379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 msgr2=0x557cdd0748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd0748e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0d54004f30 tx=0x7f0d54004f60 comp rx=0 tx=0).stop 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 -- 172.21.15.103:0/2010325379 shutdown_connections 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557cdd086f50 0x557cdd089340 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd0748e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/2010325379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd0743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d42ffd640 0 client.15948 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:47.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 -- 172.21.15.103:0/2010325379 >> 172.21.15.103:0/2010325379 conn(0x557cdd0665f0 msgr2=0x557cdd065750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:47.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 -- 172.21.15.103:0/2010325379 shutdown_connections 2024-09-05T02:06:47.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 1 -- 172.21.15.103:0/2010325379 wait complete. 2024-09-05T02:06:47.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.505+0000 7f0d6bc58740 20 client.15948 trim_cache size 0 max 16384 2024-09-05T02:06:47.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d6bc58740 1 Processor -- start 2024-09-05T02:06:47.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d6bc58740 1 -- start start 2024-09-05T02:06:47.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd069ea0 0x557cdd076310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd08e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd077450 0x557cdd08eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d641ac9c0 con 0x557cdd06a7c0 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d641ae5c0 con 0x557cdd077450 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d641ad090 con 0x557cdd069ea0 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd08e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd08e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46260/0 (socket says 172.21.15.103:46260) 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d62ffd640 1 -- 172.21.15.103:0/3112415320 learned_addr learned my addr 172.21.15.103:0/3112415320 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d627fc640 1 --2- 172.21.15.103:0/3112415320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd077450 0x557cdd08eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d627fc640 1 --2- 172.21.15.103:0/3112415320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd069ea0 0x557cdd076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d62ffd640 1 -- 172.21.15.103:0/3112415320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd069ea0 msgr2=0x557cdd076310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/3112415320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd069ea0 0x557cdd076310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d62ffd640 1 -- 172.21.15.103:0/3112415320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd077450 msgr2=0x557cdd08eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:47.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/3112415320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd077450 0x557cdd08eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.522+0000 7f0d62ffd640 1 -- 172.21.15.103:0/3112415320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cdd079b70 con 0x557cdd06a7c0 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/3112415320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd08e970 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0d54004cd0 tx=0x7f0d54009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c003d323764f3a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d597fa640 1 -- 172.21.15.103:0/3112415320 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54016040 con 0x557cdd06a7c0 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d597fa640 1 -- 172.21.15.103:0/3112415320 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d540097c0 con 0x557cdd06a7c0 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d597fa640 1 -- 172.21.15.103:0/3112415320 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d540116d0 con 0x557cdd06a7c0 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3112415320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 msgr2=0x557cdd08e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3112415320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd08e970 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0d54004cd0 tx=0x7f0d54009af0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3112415320 shutdown_connections 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3112415320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd077450 0x557cdd08eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3112415320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd08e970 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3112415320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd069ea0 0x557cdd076310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3112415320 >> 172.21.15.103:0/3112415320 conn(0x557cdd0665f0 msgr2=0x557cdd075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3112415320 shutdown_connections 2024-09-05T02:06:47.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.523+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3112415320 wait complete. 2024-09-05T02:06:47.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d6bc58740 1 -- start start 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d6bc58740 1 Processor -- start 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd07fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd07a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d6bc58740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd07a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d641adef0 con 0x557cdd06a7c0 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d641ad760 con 0x557cdd069ea0 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d6bc58740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d641aa9a0 con 0x557cdd077450 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d6bc58740 10 client.0 _ll_register_callbacks cb 0x7f0d5400eef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd07a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.524+0000 7f0d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd07a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46264/0 (socket says 172.21.15.103:46264) 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d62ffd640 1 -- 172.21.15.103:0/3400910350 learned_addr learned my addr 172.21.15.103:0/3400910350 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d627fc640 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd07fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d62ffd640 1 -- 172.21.15.103:0/3400910350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 msgr2=0x557cdd07a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd07a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d62ffd640 1 -- 172.21.15.103:0/3400910350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 msgr2=0x557cdd07fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd07fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d62ffd640 1 -- 172.21.15.103:0/3400910350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cdd07b160 con 0x557cdd06a7c0 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d62ffd640 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd07a3f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009bc0 tx=0x7f0d5400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc363ac827ec26 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d42ffd640 1 -- 172.21.15.103:0/3400910350 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54016040 con 0x557cdd06a7c0 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d42ffd640 1 -- 172.21.15.103:0/3400910350 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0d54005420 con 0x557cdd06a7c0 2024-09-05T02:06:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3400910350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d540097c0 con 0x557cdd06a7c0 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d42ffd640 1 -- 172.21.15.103:0/3400910350 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0d54005620 con 0x557cdd06a7c0 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d6bc58740 10 client.15954 fetch_fsmap learned FSMap version 35 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3400910350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557cdd0f4d30 con 0x557cdd06a7c0 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.525+0000 7f0d42ffd640 1 -- 172.21.15.103:0/3400910350 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d540097c0 con 0x557cdd06a7c0 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d42ffd640 1 -- 172.21.15.103:0/3400910350 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f0d540035f0 con 0x557cdd06a7c0 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d6bc58740 10 client.15954 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d6bc58740 10 client.15954 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d6bc58740 20 client.15954 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557cdd0f52e0 0x557cdd0f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d6bc58740 4 client.15954 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9aa7d699", "release": true, "await": true}] 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3400910350 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9aa7d699", "release": true, "await": true}) -- 0x557cdcfd70a0 con 0x557cdd0f52e0 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d627fc640 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557cdd0f52e0 0x557cdd0f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d627fc640 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557cdd0f52e0 0x557cdd0f76d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523054cc7d5e8384 server_cookie=8230548d5d3ee8d7 in_seq=0 out_seq=0 2024-09-05T02:06:47.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.526+0000 7f0d42ffd640 10 client.15954 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d42ffd640 1 -- 172.21.15.103:0/3400910350 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557cdcfd70a0 con 0x557cdd0f52e0 2024-09-05T02:06:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d42ffd640 10 client.15954 handle_command_reply: tid=0 2024-09-05T02:06:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 client.15954 shutdown 2024-09-05T02:06:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3400910350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 msgr2=0x557cdd07a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd07a3f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009bc0 tx=0x7f0d5400ecc0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3400910350 shutdown_connections 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x557cdd0f52e0 0x557cdd0f76d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cdd077450 0x557cdd07a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d42ffd640 0 client.15954 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557cdd06a7c0 0x557cdd07a3f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 --2- 172.21.15.103:0/3400910350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cdd069ea0 0x557cdd07fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3400910350 >> 172.21.15.103:0/3400910350 conn(0x557cdd0665f0 msgr2=0x557cdd06c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3400910350 shutdown_connections 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 1 -- 172.21.15.103:0/3400910350 wait complete. 2024-09-05T02:06:47.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.646+0000 7f0d6bc58740 20 client.15954 trim_cache size 0 max 16384 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 35, 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 16, 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "9aa7d699": { 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 16, 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:06:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 141.8, 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:06:47.651 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:06:47.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 -- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d500a2b80 msgr2=0x7f0d500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:47.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d500a2b80 0x7f0d500a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0b8d60 tx=0x7f0d5c08e040 comp rx=0 tx=0).stop 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 -- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64106a80 msgr2=0x7f0d6419bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64106a80 0x7f0d6419bb00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c005cc0 tx=0x7f0d5c003c20 comp rx=0 tx=0).stop 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 -- 172.21.15.103:0/395890101 shutdown_connections 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d64106a80 0x7f0d6419bb00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0d500a2b80 0x7f0d500a5040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d64101420 0x7f0d64197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 --2- 172.21.15.103:0/395890101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d64100a80 0x7f0d64197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 -- 172.21.15.103:0/395890101 >> 172.21.15.103:0/395890101 conn(0x7f0d640fc7a0 msgr2=0x7f0d640fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 -- 172.21.15.103:0/395890101 shutdown_connections 2024-09-05T02:06:47.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:06:47.647+0000 7f0d6987f640 1 -- 172.21.15.103:0/395890101 wait complete. 2024-09-05T02:06:48.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:48 smithi103 ceph-mon[30902]: pgmap v481: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 80 KiB/s wr, 15 op/s 2024-09-05T02:06:48.611 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9aa7d699', total seconds elapsed: 27.0 2024-09-05T02:06:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:48 smithi167 ceph-mon[35122]: pgmap v481: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 80 KiB/s wr, 15 op/s 2024-09-05T02:06:48.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:48 smithi139 ceph-mon[34262]: pgmap v481: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 80 KiB/s wr, 15 op/s 2024-09-05T02:06:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:50 smithi167 ceph-mon[35122]: pgmap v482: 129 pgs: 129 active+clean; 234 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 142 KiB/s wr, 20 op/s 2024-09-05T02:06:50.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:50 smithi139 ceph-mon[34262]: pgmap v482: 129 pgs: 129 active+clean; 234 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 142 KiB/s wr, 20 op/s 2024-09-05T02:06:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:50 smithi103 ceph-mon[30902]: pgmap v482: 129 pgs: 129 active+clean; 234 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 142 KiB/s wr, 20 op/s 2024-09-05T02:06:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:52 smithi167 ceph-mon[35122]: pgmap v483: 129 pgs: 129 active+clean; 234 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 21 op/s 2024-09-05T02:06:52.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:52 smithi139 ceph-mon[34262]: pgmap v483: 129 pgs: 129 active+clean; 234 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 21 op/s 2024-09-05T02:06:52.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:52 smithi103 ceph-mon[30902]: pgmap v483: 129 pgs: 129 active+clean; 234 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 21 op/s 2024-09-05T02:06:53.875 INFO:tasks.workunit.client.0.smithi103.stdout: 47 7322 104 4087 38 2060 100 2024-09-05T02:06:54.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:54 smithi167 ceph-mon[35122]: pgmap v484: 129 pgs: 129 active+clean; 237 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 362 KiB/s wr, 58 op/s 2024-09-05T02:06:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:54 smithi139 ceph-mon[34262]: pgmap v484: 129 pgs: 129 active+clean; 237 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 362 KiB/s wr, 58 op/s 2024-09-05T02:06:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:54 smithi103 ceph-mon[30902]: pgmap v484: 129 pgs: 129 active+clean; 237 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 362 KiB/s wr, 58 op/s 2024-09-05T02:06:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.530+0000 7f2c4c227640 1 -- 172.21.15.103:0/322298712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c44102d50 msgr2=0x7f2c441031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.530+0000 7f2c4c227640 1 --2- 172.21.15.103:0/322298712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c44102d50 0x7f2c441031d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009900 tx=0x7f2c3402a410 comp rx=0 tx=0).stop 2024-09-05T02:06:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.530+0000 7f2c4c227640 1 -- 172.21.15.103:0/322298712 shutdown_connections 2024-09-05T02:06:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.530+0000 7f2c4c227640 1 --2- 172.21.15.103:0/322298712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c44103710 0x7f2c44109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.530+0000 7f2c4c227640 1 --2- 172.21.15.103:0/322298712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c44102d50 0x7f2c441031d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.530+0000 7f2c4c227640 1 --2- 172.21.15.103:0/322298712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c44101b60 0x7f2c44101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.530+0000 7f2c4c227640 1 -- 172.21.15.103:0/322298712 >> 172.21.15.103:0/322298712 conn(0x7f2c440fd2e0 msgr2=0x7f2c440ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.530+0000 7f2c4c227640 1 -- 172.21.15.103:0/322298712 shutdown_connections 2024-09-05T02:06:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.530+0000 7f2c4c227640 1 -- 172.21.15.103:0/322298712 wait complete. 2024-09-05T02:06:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c4c227640 1 Processor -- start 2024-09-05T02:06:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c4c227640 1 -- start start 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c4c227640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c44101b60 0x7f2c4419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c4c227640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c44103710 0x7f2c4419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c4c227640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c4419f6a0 0x7f2c441a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c4c227640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410d6e0 con 0x7f2c4419f6a0 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c4c227640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410d560 con 0x7f2c44103710 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c4c227640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410d860 con 0x7f2c44101b60 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c49fc3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c44101b60 0x7f2c4419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c4a7c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c4419f6a0 0x7f2c441a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c49fc3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c44101b60 0x7f2c4419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:60738/0 (socket says 172.21.15.103:60738) 2024-09-05T02:06:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.531+0000 7f2c49fc3640 1 -- 172.21.15.103:0/646253077 learned_addr learned my addr 172.21.15.103:0/646253077 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c497c2640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c44103710 0x7f2c4419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c49fc3640 1 -- 172.21.15.103:0/646253077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c44103710 msgr2=0x7f2c4419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c49fc3640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c44103710 0x7f2c4419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c49fc3640 1 -- 172.21.15.103:0/646253077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c4419f6a0 msgr2=0x7f2c441a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c49fc3640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c4419f6a0 0x7f2c441a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c49fc3640 1 -- 172.21.15.103:0/646253077 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c441a2100 con 0x7f2c44101b60 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c497c2640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c44103710 0x7f2c4419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c49fc3640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c44101b60 0x7f2c4419a100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000e9c0 tx=0x7f2c4000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c3affd640 1 -- 172.21.15.103:0/646253077 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000cd80 con 0x7f2c44101b60 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c4c227640 1 -- 172.21.15.103:0/646253077 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c441a23f0 con 0x7f2c44101b60 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c3affd640 1 -- 172.21.15.103:0/646253077 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c400035f0 con 0x7f2c44101b60 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.532+0000 7f2c3affd640 1 -- 172.21.15.103:0/646253077 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40010690 con 0x7f2c44101b60 2024-09-05T02:06:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.533+0000 7f2c4c227640 1 -- 172.21.15.103:0/646253077 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c441a2960 con 0x7f2c44101b60 2024-09-05T02:06:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c4efc0740 1 Processor -- start 2024-09-05T02:06:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c3affd640 1 -- 172.21.15.103:0/646253077 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2c400037a0 con 0x7f2c44101b60 2024-09-05T02:06:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c4efc0740 1 -- start start 2024-09-05T02:06:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed04da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed09da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed0a370 0x55d02ed0c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c4efc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d02ec682f0 con 0x55d02ed0a370 2024-09-05T02:06:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c4efc0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d02ec10270 con 0x55d02ed052e0 2024-09-05T02:06:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c4efc0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d02ec642b0 con 0x55d02ed049a0 2024-09-05T02:06:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c49fc3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed09da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c49fc3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed09da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:54404/0 (socket says 172.21.15.103:54404) 2024-09-05T02:06:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c49fc3640 1 -- 172.21.15.103:0/2800354984 learned_addr learned my addr 172.21.15.103:0/2800354984 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c497c2640 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed0a370 0x55d02ed0c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.536+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed04da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.537+0000 7f2c4a7c4640 1 -- 172.21.15.103:0/2800354984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 msgr2=0x55d02ed09da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.537+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed09da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.537+0000 7f2c4a7c4640 1 -- 172.21.15.103:0/2800354984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed0a370 msgr2=0x55d02ed0c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.537+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed0a370 0x55d02ed0c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.537+0000 7f2c4a7c4640 1 -- 172.21.15.103:0/2800354984 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d02ec0f340 con 0x55d02ed049a0 2024-09-05T02:06:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.537+0000 7f2c3affd640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2c300a2cb0 0x7f2c300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.537+0000 7f2c497c2640 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed0a370 0x55d02ed0c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.537+0000 7f2c49fc3640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2c300a2cb0 0x7f2c300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.537+0000 7f2c3affd640 1 -- 172.21.15.103:0/646253077 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c400cf790 con 0x7f2c44101b60 2024-09-05T02:06:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.538+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed04da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009f00 tx=0x7f2c3c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=246300242db11f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.538+0000 7f2c49fc3640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2c300a2cb0 0x7f2c300a5170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c400980a0 tx=0x7f2c400d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.538+0000 7f2c2ffff640 1 -- 172.21.15.103:0/2800354984 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c004d20 con 0x55d02ed049a0 2024-09-05T02:06:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.538+0000 7f2c2ffff640 1 -- 172.21.15.103:0/2800354984 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c3c004030 con 0x55d02ed049a0 2024-09-05T02:06:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.538+0000 7f2c2ffff640 1 -- 172.21.15.103:0/2800354984 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c005280 con 0x55d02ed049a0 2024-09-05T02:06:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 -- 172.21.15.103:0/2800354984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 msgr2=0x55d02ed04da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed04da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009f00 tx=0x7f2c3c011810 comp rx=0 tx=0).stop 2024-09-05T02:06:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 -- 172.21.15.103:0/2800354984 shutdown_connections 2024-09-05T02:06:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed0a370 0x55d02ed0c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed09da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/2800354984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed04da0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 -- 172.21.15.103:0/2800354984 >> 172.21.15.103:0/2800354984 conn(0x55d02ed00010 msgr2=0x55d02ed00410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 -- 172.21.15.103:0/2800354984 shutdown_connections 2024-09-05T02:06:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 -- 172.21.15.103:0/2800354984 wait complete. 2024-09-05T02:06:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.539+0000 7f2c4efc0740 1 -- start start 2024-09-05T02:06:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4efc0740 1 Processor -- start 2024-09-05T02:06:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed049a0 0x55d02ed0eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed0f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 0x55d02ed160e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4efc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d02ed0d340 con 0x55d02ed052e0 2024-09-05T02:06:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4efc0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d02ebf1290 con 0x55d02ed049a0 2024-09-05T02:06:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4efc0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d02ed0d4c0 con 0x55d02ed0a370 2024-09-05T02:06:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4efc0740 10 client.0 _ll_register_callbacks cb 0x55d02ec22140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c497c2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed0f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4a7c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed049a0 0x55d02ed0eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c497c2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed0f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42824/0 (socket says 172.21.15.103:42824) 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c497c2640 1 -- 172.21.15.103:0/3461115044 learned_addr learned my addr 172.21.15.103:0/3461115044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.540+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 0x55d02ed160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c497c2640 1 -- 172.21.15.103:0/3461115044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 msgr2=0x55d02ed160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 0x55d02ed160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c497c2640 1 -- 172.21.15.103:0/3461115044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed049a0 msgr2=0x55d02ed0eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed049a0 0x55d02ed0eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c497c2640 1 -- 172.21.15.103:0/3461115044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d02e97b210 con 0x55d02ed052e0 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed0f0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c340044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5701ea6ba184256b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3461115044 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34040070 con 0x55d02ed052e0 2024-09-05T02:06:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3461115044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c3c004030 con 0x55d02ed052e0 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3461115044 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c340054d0 con 0x55d02ed052e0 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3461115044 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2c3404c050 con 0x55d02ed052e0 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c4efc0740 10 client.15966 fetch_fsmap learned FSMap version 35 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.541+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3461115044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d02ea47620 con 0x55d02ed052e0 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.542+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3461115044 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c3c004030 con 0x55d02ed052e0 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.542+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3461115044 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f2c34003810 con 0x55d02ed052e0 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.542+0000 7f2c4efc0740 10 client.15966 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.542+0000 7f2c4efc0740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.542+0000 7f2c4efc0740 20 client.15966 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.542+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d02ed1a820 0x55d02ed1cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.542+0000 7f2c4efc0740 4 client.15966 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.542+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3461115044 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d02ec75a20 con 0x55d02ed1a820 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.542+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d02ed1a820 0x55d02ed1cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.543+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d02ed1a820 0x55d02ed1cc10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9006064cb72a937 server_cookie=197354cf5fbd0e3d in_seq=0 out_seq=0 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.543+0000 7f2c2d7fa640 10 client.15966 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3461115044 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d02ec75a20 con 0x55d02ed1a820 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c2d7fa640 10 client.15966 handle_command_reply: tid=0 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 client.15966 shutdown 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3461115044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 msgr2=0x55d02ed0f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed0f0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c340044d0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3461115044 shutdown_connections 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d02ed1a820 0x55d02ed1cc10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 0x55d02ed160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed0f0a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3461115044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed049a0 0x55d02ed0eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3461115044 >> 172.21.15.103:0/3461115044 conn(0x55d02ed00010 msgr2=0x55d02ed01820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c2d7fa640 0 client.15966 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3461115044 shutdown_connections 2024-09-05T02:06:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3461115044 wait complete. 2024-09-05T02:06:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.544+0000 7f2c4efc0740 20 client.15966 trim_cache size 0 max 16384 2024-09-05T02:06:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.573+0000 7f2c4efc0740 1 Processor -- start 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.573+0000 7f2c4efc0740 1 -- start start 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed049a0 0x55d02ecff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 0x55d02ed21510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4efc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441acf20 con 0x55d02ed049a0 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4efc0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441aeb20 con 0x55d02ed052e0 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4efc0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441ad5f0 con 0x55d02ed0a370 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4a7c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed049a0 0x55d02ecff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c497c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4a7c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed049a0 0x55d02ecff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42826/0 (socket says 172.21.15.103:42826) 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c497c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:54426/0 (socket says 172.21.15.103:54426) 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4a7c4640 1 -- 172.21.15.103:0/3096810311 learned_addr learned my addr 172.21.15.103:0/3096810311 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/3096810311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 0x55d02ed21510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c497c2640 1 -- 172.21.15.103:0/3096810311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 msgr2=0x55d02ed21510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3096810311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 0x55d02ed21510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c497c2640 1 -- 172.21.15.103:0/3096810311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed049a0 msgr2=0x55d02ecff0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3096810311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed049a0 0x55d02ecff0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c497c2640 1 -- 172.21.15.103:0/3096810311 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d02ed057c0 con 0x55d02ed052e0 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/3096810311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed049a0 0x55d02ecff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.574+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3096810311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed160a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c3402aa00 tx=0x7f2c34005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6be789b2ac9eca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.575+0000 7f2c2f7fe640 1 -- 172.21.15.103:0/3096810311 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34055070 con 0x55d02ed052e0 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.575+0000 7f2c2f7fe640 1 -- 172.21.15.103:0/3096810311 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c34031580 con 0x55d02ed052e0 2024-09-05T02:06:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.575+0000 7f2c2f7fe640 1 -- 172.21.15.103:0/3096810311 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34040070 con 0x55d02ed052e0 2024-09-05T02:06:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.575+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3096810311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 msgr2=0x55d02ed160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.575+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3096810311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed160a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c3402aa00 tx=0x7f2c34005fd0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.575+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3096810311 shutdown_connections 2024-09-05T02:06:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.575+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3096810311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed0a370 0x55d02ed21510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.576+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3096810311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed052e0 0x55d02ed160a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.576+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3096810311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed049a0 0x55d02ecff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.576+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3096810311 >> 172.21.15.103:0/3096810311 conn(0x55d02ed00010 msgr2=0x55d02ecfe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.576+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3096810311 shutdown_connections 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.576+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3096810311 wait complete. 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.576+0000 7f2c4efc0740 1 -- start start 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.576+0000 7f2c4efc0740 1 Processor -- start 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed13500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed13a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4efc0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed0a370 0x55d02ed14a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4a7c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed13500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4a7c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed13500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:60776/0 (socket says 172.21.15.103:60776) 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4a7c4640 1 -- 172.21.15.103:0/3239258989 learned_addr learned my addr 172.21.15.103:0/3239258989 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:06:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441ae450 con 0x55d02ed052e0 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441adcc0 con 0x55d02ed0a370 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441aaf00 con 0x55d02ed049a0 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed13a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed0a370 0x55d02ed14a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c497c2640 1 -- 172.21.15.103:0/3239258989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 msgr2=0x55d02ed13500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed13500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c497c2640 1 -- 172.21.15.103:0/3239258989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed0a370 msgr2=0x55d02ed14a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed0a370 0x55d02ed14a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c497c2640 1 -- 172.21.15.103:0/3239258989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d02ed15010 con 0x55d02ed052e0 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4efc0740 10 client.0 _ll_register_callbacks cb 0x7f2c3c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c34031580 con 0x55d02ed052e0 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.577+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed13500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:06:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.578+0000 7f2c497c2640 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed13a40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2c34001eb0 tx=0x7f2c340044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcff01951f8c1a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.578+0000 7f2c2d7fa640 10 client.15972 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.578+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3239258989 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3405e060 con 0x55d02ed052e0 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.578+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3239258989 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c340035f0 con 0x55d02ed052e0 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.578+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3239258989 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2c34040070 con 0x55d02ed052e0 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.578+0000 7f2c4efc0740 10 client.15972 fetch_fsmap learned FSMap version 35 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.578+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=35}) -- 0x55d02ed16210 con 0x55d02ed052e0 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.579+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3239258989 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f2c34058030 con 0x55d02ed052e0 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.579+0000 7f2c4efc0740 10 client.15972 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.579+0000 7f2c4efc0740 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.579+0000 7f2c4efc0740 20 client.15972 populate_metadata read hostname 'smithi103' 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.579+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d02ed85970 0x55d02ed87d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:06:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.579+0000 7f2c4efc0740 4 client.15972 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:06:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.579+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d02ec6fd70 con 0x55d02ed85970 2024-09-05T02:06:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.579+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d02ed85970 0x55d02ed87d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:06:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.580+0000 7f2c2d7fa640 10 client.15972 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.580+0000 7f2c4a7c4640 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d02ed85970 0x55d02ed87d60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=191e833bdff9fec3 server_cookie=3a9a92d609d336a in_seq=0 out_seq=0 2024-09-05T02:06:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.640+0000 7f2c2d7fa640 1 -- 172.21.15.103:0/3239258989 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d02ec6fd70 con 0x55d02ed85970 2024-09-05T02:06:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.640+0000 7f2c2d7fa640 10 client.15972 handle_command_reply: tid=0 2024-09-05T02:06:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.640+0000 7f2c4efc0740 1 client.15972 shutdown 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.640+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 msgr2=0x55d02ed13a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.640+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed13a40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2c34001eb0 tx=0x7f2c340044b0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.640+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 shutdown_connections 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.640+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d02ed85970 0x55d02ed87d60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.640+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02ed0a370 0x55d02ed14a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.641+0000 7f2c2d7fa640 0 client.15972 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.641+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d02ed052e0 0x55d02ed13a40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.641+0000 7f2c4efc0740 1 --2- 172.21.15.103:0/3239258989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d02ed049a0 0x55d02ed13500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.641+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 >> 172.21.15.103:0/3239258989 conn(0x55d02ed00010 msgr2=0x55d02ecff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.641+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 shutdown_connections 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.641+0000 7f2c4efc0740 1 -- 172.21.15.103:0/3239258989 wait complete. 2024-09-05T02:06:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.641+0000 7f2c4efc0740 20 client.15972 trim_cache size 0 max 16384 2024-09-05T02:06:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:06:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:06:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:06:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:06:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 -- 172.21.15.103:0/646253077 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2c300a2cb0 msgr2=0x7f2c300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2c300a2cb0 0x7f2c300a5170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c400980a0 tx=0x7f2c400d5550 comp rx=0 tx=0).stop 2024-09-05T02:06:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 -- 172.21.15.103:0/646253077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c44101b60 msgr2=0x7f2c4419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:06:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c44101b60 0x7f2c4419a100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000e9c0 tx=0x7f2c4000ee80 comp rx=0 tx=0).stop 2024-09-05T02:06:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 -- 172.21.15.103:0/646253077 shutdown_connections 2024-09-05T02:06:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c4419f6a0 0x7f2c441a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c44103710 0x7f2c4419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f2c300a2cb0 0x7f2c300a5170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 --2- 172.21.15.103:0/646253077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c44101b60 0x7f2c4419a100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:06:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 -- 172.21.15.103:0/646253077 >> 172.21.15.103:0/646253077 conn(0x7f2c440fd2e0 msgr2=0x7f2c44108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:06:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.642+0000 7f2c4c227640 1 -- 172.21.15.103:0/646253077 shutdown_connections 2024-09-05T02:06:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:06:55.643+0000 7f2c4c227640 1 -- 172.21.15.103:0/646253077 wait complete. 2024-09-05T02:06:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:56 smithi167 ceph-mon[35122]: pgmap v485: 129 pgs: 129 active+clean; 241 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 765 KiB/s wr, 105 op/s 2024-09-05T02:06:56.644 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:56 smithi103 ceph-mon[30902]: pgmap v485: 129 pgs: 129 active+clean; 241 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 765 KiB/s wr, 105 op/s 2024-09-05T02:06:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:56 smithi139 ceph-mon[34262]: pgmap v485: 129 pgs: 129 active+clean; 241 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 765 KiB/s wr, 105 op/s 2024-09-05T02:06:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:06:58 smithi167 ceph-mon[35122]: pgmap v486: 129 pgs: 129 active+clean; 242 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 862 KiB/s wr, 115 op/s 2024-09-05T02:06:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:06:58 smithi139 ceph-mon[34262]: pgmap v486: 129 pgs: 129 active+clean; 242 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 862 KiB/s wr, 115 op/s 2024-09-05T02:06:58.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:06:58 smithi103 ceph-mon[30902]: pgmap v486: 129 pgs: 129 active+clean; 242 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 862 KiB/s wr, 115 op/s 2024-09-05T02:07:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:00 smithi167 ceph-mon[35122]: pgmap v487: 129 pgs: 129 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-05T02:07:00.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:00 smithi139 ceph-mon[34262]: pgmap v487: 129 pgs: 129 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-05T02:07:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:00 smithi103 ceph-mon[30902]: pgmap v487: 129 pgs: 129 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-05T02:07:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:02 smithi167 ceph-mon[35122]: pgmap v488: 129 pgs: 129 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-09-05T02:07:02.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:02 smithi139 ceph-mon[34262]: pgmap v488: 129 pgs: 129 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-09-05T02:07:02.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:02 smithi103 ceph-mon[30902]: pgmap v488: 129 pgs: 129 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-09-05T02:07:03.875 INFO:tasks.workunit.client.0.smithi103.stdout: 48 12086 50 7905 127 3851 108 2024-09-05T02:07:04.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:04 smithi103 ceph-mon[30902]: pgmap v489: 129 pgs: 129 active+clean; 253 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-09-05T02:07:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:04 smithi167 ceph-mon[35122]: pgmap v489: 129 pgs: 129 active+clean; 253 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-09-05T02:07:04.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:04 smithi139 ceph-mon[34262]: pgmap v489: 129 pgs: 129 active+clean; 253 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-09-05T02:07:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:06 smithi167 ceph-mon[35122]: pgmap v490: 129 pgs: 129 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 244 op/s 2024-09-05T02:07:06.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:06 smithi139 ceph-mon[34262]: pgmap v490: 129 pgs: 129 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 244 op/s 2024-09-05T02:07:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:06 smithi103 ceph-mon[30902]: pgmap v490: 129 pgs: 129 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 244 op/s 2024-09-05T02:07:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:08 smithi167 ceph-mon[35122]: pgmap v491: 129 pgs: 129 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-09-05T02:07:08.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:08 smithi139 ceph-mon[34262]: pgmap v491: 129 pgs: 129 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-09-05T02:07:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:08 smithi103 ceph-mon[30902]: pgmap v491: 129 pgs: 129 active+clean; 257 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-09-05T02:07:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.764+0000 7f84b35d8640 1 -- 172.21.15.103:0/2950160023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac101c50 msgr2=0x7f84ac1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.764+0000 7f84b35d8640 1 --2- 172.21.15.103:0/2950160023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac101c50 0x7f84ac1020d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f849c009900 tx=0x7f849c02a410 comp rx=0 tx=0).stop 2024-09-05T02:07:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.764+0000 7f84b35d8640 1 -- 172.21.15.103:0/2950160023 shutdown_connections 2024-09-05T02:07:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.764+0000 7f84b35d8640 1 --2- 172.21.15.103:0/2950160023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac102610 0x7f84ac108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.764+0000 7f84b35d8640 1 --2- 172.21.15.103:0/2950160023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac101c50 0x7f84ac1020d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.764+0000 7f84b35d8640 1 --2- 172.21.15.103:0/2950160023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84ac100a60 0x7f84ac100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.764+0000 7f84b35d8640 1 -- 172.21.15.103:0/2950160023 >> 172.21.15.103:0/2950160023 conn(0x7f84ac0fc1c0 msgr2=0x7f84ac0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.764+0000 7f84b35d8640 1 -- 172.21.15.103:0/2950160023 shutdown_connections 2024-09-05T02:07:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.764+0000 7f84b35d8640 1 -- 172.21.15.103:0/2950160023 wait complete. 2024-09-05T02:07:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.765+0000 7f84b35d8640 1 Processor -- start 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b35d8640 1 -- start start 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b35d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac100a60 0x7f84ac199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b35d8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84ac101c50 0x7f84ac19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b35d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac102610 0x7f84ac1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b35d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac10b460 con 0x7f84ac102610 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b35d8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac10b2e0 con 0x7f84ac100a60 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b35d8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac10b5e0 con 0x7f84ac101c50 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b1b75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac102610 0x7f84ac1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b1b75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac102610 0x7f84ac1a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51936/0 (socket says 172.21.15.103:51936) 2024-09-05T02:07:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b1b75640 1 -- 172.21.15.103:0/846458809 learned_addr learned my addr 172.21.15.103:0/846458809 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.766+0000 7f84b1374640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac100a60 0x7f84ac199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84b1b75640 1 -- 172.21.15.103:0/846458809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84ac101c50 msgr2=0x7f84ac19a460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T02:07:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84b1b75640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84ac101c50 0x7f84ac19a460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84b1b75640 1 -- 172.21.15.103:0/846458809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac100a60 msgr2=0x7f84ac199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84b1b75640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac100a60 0x7f84ac199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84b1b75640 1 -- 172.21.15.103:0/846458809 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac1a1b40 con 0x7f84ac102610 2024-09-05T02:07:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84b1374640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac100a60 0x7f84ac199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84b1b75640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac102610 0x7f84ac1a14d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f84a400eb10 tx=0x7f84a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84a27fc640 1 -- 172.21.15.103:0/846458809 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4004b20 con 0x7f84ac102610 2024-09-05T02:07:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84b35d8640 1 -- 172.21.15.103:0/846458809 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ac1a1e30 con 0x7f84ac102610 2024-09-05T02:07:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84b35d8640 1 -- 172.21.15.103:0/846458809 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ac1a22f0 con 0x7f84ac102610 2024-09-05T02:07:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84a27fc640 1 -- 172.21.15.103:0/846458809 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a4004cc0 con 0x7f84ac102610 2024-09-05T02:07:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.767+0000 7f84a27fc640 1 -- 172.21.15.103:0/846458809 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a40106c0 con 0x7f84ac102610 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.771+0000 7f84a27fc640 1 -- 172.21.15.103:0/846458809 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f84a4010860 con 0x7f84ac102610 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.771+0000 7f84b6449740 1 Processor -- start 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.771+0000 7f84b6449740 1 -- start start 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.771+0000 7f84b6449740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e189a0 0x5564b8e18da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.771+0000 7f84b6449740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 0x5564b8e1dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.771+0000 7f84b6449740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e1e370 0x5564b8e20830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b6449740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5564b8d7c2f0 con 0x5564b8e1e370 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b6449740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5564b8d24270 con 0x5564b8e192e0 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b6449740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5564b8d782b0 con 0x5564b8e189a0 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b0b73640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e189a0 0x5564b8e18da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b0b73640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e189a0 0x5564b8e18da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:56078/0 (socket says 172.21.15.103:56078) 2024-09-05T02:07:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b0b73640 1 -- 172.21.15.103:0/2984272008 learned_addr learned my addr 172.21.15.103:0/2984272008 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b0b73640 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e1e370 0x5564b8e20830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b1374640 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 0x5564b8e1dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b1374640 1 -- 172.21.15.103:0/2984272008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e189a0 msgr2=0x5564b8e18da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b1374640 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e189a0 0x5564b8e18da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b1374640 1 -- 172.21.15.103:0/2984272008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e1e370 msgr2=0x5564b8e20830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b1374640 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e1e370 0x5564b8e20830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b1374640 1 -- 172.21.15.103:0/2984272008 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564b8d23340 con 0x5564b8e192e0 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b0b73640 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e1e370 0x5564b8e20830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.772+0000 7f84b1374640 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 0x5564b8e1dda0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009fa0 tx=0x7f84a80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66cc44c13e3c547a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.773+0000 7f84977fe640 1 -- 172.21.15.103:0/2984272008 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004310 con 0x5564b8e192e0 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.773+0000 7f84977fe640 1 -- 172.21.15.103:0/2984272008 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a80044b0 con 0x5564b8e192e0 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.773+0000 7f84977fe640 1 -- 172.21.15.103:0/2984272008 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a80035f0 con 0x5564b8e192e0 2024-09-05T02:07:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.773+0000 7f84a27fc640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f84980a2cb0 0x7f84980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.773+0000 7f84b0b73640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f84980a2cb0 0x7f84980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 -- 172.21.15.103:0/2984272008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 msgr2=0x5564b8e1dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 0x5564b8e1dda0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009fa0 tx=0x7f84a80118d0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 -- 172.21.15.103:0/2984272008 shutdown_connections 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e1e370 0x5564b8e20830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 0x5564b8e1dda0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 --2- 172.21.15.103:0/2984272008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e189a0 0x5564b8e18da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 -- 172.21.15.103:0/2984272008 >> 172.21.15.103:0/2984272008 conn(0x5564b8e14010 msgr2=0x5564b8e14410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b0b73640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f84980a2cb0 0x7f84980a5170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f849c009230 tx=0x7f849c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84a27fc640 1 -- 172.21.15.103:0/846458809 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f84a4005630 con 0x7f84ac102610 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 -- 172.21.15.103:0/2984272008 shutdown_connections 2024-09-05T02:07:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 -- 172.21.15.103:0/2984272008 wait complete. 2024-09-05T02:07:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.774+0000 7f84b6449740 1 -- start start 2024-09-05T02:07:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.775+0000 7f84b6449740 1 Processor -- start 2024-09-05T02:07:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.775+0000 7f84b6449740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e22ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.775+0000 7f84b6449740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e192e0 0x5564b8e230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.775+0000 7f84b6449740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e1e370 0x5564b8e2a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.775+0000 7f84b1374640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e22ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.775+0000 7f84b1374640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e22ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51956/0 (socket says 172.21.15.103:51956) 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b1374640 1 -- 172.21.15.103:0/433586916 learned_addr learned my addr 172.21.15.103:0/433586916 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b0b73640 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e192e0 0x5564b8e230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5564b8e21340 con 0x5564b8e189a0 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5564b8d05290 con 0x5564b8e1e370 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5564b8e214c0 con 0x5564b8e192e0 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b1b75640 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e1e370 0x5564b8e2a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b6449740 10 client.0 _ll_register_callbacks cb 0x5564b8d36140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b1b75640 1 -- 172.21.15.103:0/433586916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e192e0 msgr2=0x5564b8e230e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b1b75640 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e192e0 0x5564b8e230e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b1b75640 1 -- 172.21.15.103:0/433586916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 msgr2=0x5564b8e22ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b1b75640 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e22ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b1b75640 1 -- 172.21.15.103:0/433586916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564b8a8f210 con 0x5564b8e1e370 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.776+0000 7f84b1374640 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e22ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.777+0000 7f84b1b75640 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e1e370 0x5564b8e2a120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f84a409ccb0 tx=0x7f84a409cfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd78b48f094c2de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.777+0000 7f8494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.777+0000 7f8494ff9640 1 -- 172.21.15.103:0/433586916 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a40b5020 con 0x5564b8e1e370 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.777+0000 7f8494ff9640 1 -- 172.21.15.103:0/433586916 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f84a409c400 con 0x5564b8e1e370 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.777+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a80044b0 con 0x5564b8e1e370 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.777+0000 7f8494ff9640 1 -- 172.21.15.103:0/433586916 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f84a40a5c20 con 0x5564b8e1e370 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.777+0000 7f84b6449740 10 client.25412 fetch_fsmap learned FSMap version 35 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.777+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5564b8b5b620 con 0x5564b8e1e370 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.777+0000 7f8494ff9640 1 -- 172.21.15.103:0/433586916 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a80044b0 con 0x5564b8e1e370 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f8494ff9640 1 -- 172.21.15.103:0/433586916 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f84a4086160 con 0x5564b8e1e370 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f84b6449740 10 client.25412 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f84b6449740 10 client.25412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:07:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f84b6449740 20 client.25412 populate_metadata read hostname 'smithi103' 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f84b6449740 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5564b8e2e890 0x5564b8e30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f84b6449740 4 client.25412 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564b8d89a20 con 0x5564b8e2e890 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f84b1374640 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5564b8e2e890 0x5564b8e30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f84b1374640 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5564b8e2e890 0x5564b8e30d30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=173b640e1b5e6f87 server_cookie=dac5f9ef0c22fcfb in_seq=0 out_seq=0 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.778+0000 7f8494ff9640 10 client.25412 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f8494ff9640 1 -- 172.21.15.103:0/433586916 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564b8d89a20 con 0x5564b8e2e890 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f8494ff9640 10 client.25412 handle_command_reply: tid=0 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f84b6449740 1 client.25412 shutdown 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e1e370 msgr2=0x5564b8e2a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f84b6449740 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e1e370 0x5564b8e2a120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f84a409ccb0 tx=0x7f84a409cfb0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 shutdown_connections 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f84b6449740 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5564b8e2e890 0x5564b8e30d30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f84b6449740 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e1e370 0x5564b8e2a120 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f84b6449740 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e192e0 0x5564b8e230e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f84b6449740 1 --2- 172.21.15.103:0/433586916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e22ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 >> 172.21.15.103:0/433586916 conn(0x5564b8e14010 msgr2=0x5564b8e15820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.779+0000 7f8494ff9640 0 client.25412 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.780+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 shutdown_connections 2024-09-05T02:07:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.780+0000 7f84b6449740 1 -- 172.21.15.103:0/433586916 wait complete. 2024-09-05T02:07:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.780+0000 7f84b6449740 20 client.25412 trim_cache size 0 max 16384 2024-09-05T02:07:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b6449740 1 Processor -- start 2024-09-05T02:07:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b6449740 1 -- start start 2024-09-05T02:07:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b6449740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e189a0 0x5564b8e130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b6449740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e192e0 0x5564b8e2a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b6449740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 0x5564b8e15400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b1374640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e189a0 0x5564b8e130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b1374640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e189a0 0x5564b8e130b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:53596/0 (socket says 172.21.15.103:53596) 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b0b73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e192e0 0x5564b8e2a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b0b73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e192e0 0x5564b8e2a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51960/0 (socket says 172.21.15.103:51960) 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b1374640 1 -- 172.21.15.103:0/3357124492 learned_addr learned my addr 172.21.15.103:0/3357124492 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b6449740 1 -- 172.21.15.103:0/3357124492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f849000fc50 con 0x5564b8e192e0 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b6449740 1 -- 172.21.15.103:0/3357124492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8490011850 con 0x5564b8e189a0 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.807+0000 7f84b6449740 1 -- 172.21.15.103:0/3357124492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8490010320 con 0x5564b8e1e370 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.808+0000 7f84b1b75640 1 --2- 172.21.15.103:0/3357124492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 0x5564b8e15400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.808+0000 7f84b1374640 1 -- 172.21.15.103:0/3357124492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 msgr2=0x5564b8e15400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.808+0000 7f84b1374640 1 --2- 172.21.15.103:0/3357124492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 0x5564b8e15400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.808+0000 7f84b1374640 1 -- 172.21.15.103:0/3357124492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e192e0 msgr2=0x5564b8e2a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.808+0000 7f84b1374640 1 --2- 172.21.15.103:0/3357124492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e192e0 0x5564b8e2a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.808+0000 7f84b1374640 1 -- 172.21.15.103:0/3357124492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564b8e197c0 con 0x5564b8e189a0 2024-09-05T02:07:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.808+0000 7f84b0b73640 1 --2- 172.21.15.103:0/3357124492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e192e0 0x5564b8e2a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.808+0000 7f84b1374640 1 --2- 172.21.15.103:0/3357124492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e189a0 0x5564b8e130b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5564b8e220d0 tx=0x7f84a8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d07c0b1fbda8189 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.808+0000 7f8496ffd640 1 -- 172.21.15.103:0/3357124492 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8031070 con 0x5564b8e189a0 2024-09-05T02:07:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.809+0000 7f8496ffd640 1 -- 172.21.15.103:0/3357124492 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a8004490 con 0x5564b8e189a0 2024-09-05T02:07:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.809+0000 7f8496ffd640 1 -- 172.21.15.103:0/3357124492 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a801d040 con 0x5564b8e189a0 2024-09-05T02:07:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 -- 172.21.15.103:0/3357124492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e189a0 msgr2=0x5564b8e130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 --2- 172.21.15.103:0/3357124492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e189a0 0x5564b8e130b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5564b8e220d0 tx=0x7f84a8009310 comp rx=0 tx=0).stop 2024-09-05T02:07:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 -- 172.21.15.103:0/3357124492 shutdown_connections 2024-09-05T02:07:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 --2- 172.21.15.103:0/3357124492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 0x5564b8e15400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 --2- 172.21.15.103:0/3357124492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e192e0 0x5564b8e2a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 --2- 172.21.15.103:0/3357124492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e189a0 0x5564b8e130b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 -- 172.21.15.103:0/3357124492 >> 172.21.15.103:0/3357124492 conn(0x5564b8e14010 msgr2=0x5564b8e129a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 -- 172.21.15.103:0/3357124492 shutdown_connections 2024-09-05T02:07:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 -- 172.21.15.103:0/3357124492 wait complete. 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 -- start start 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.810+0000 7f84b6449740 1 Processor -- start 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b6449740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e27000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b6449740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 0x5564b8e27540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b6449740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 0x5564b8e97600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b6449740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8490011180 con 0x5564b8e189a0 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b6449740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84900109f0 con 0x5564b8e192e0 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b6449740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f849000dc30 con 0x5564b8e1e370 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b6449740 10 client.0 _ll_register_callbacks cb 0x7f84a4004ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1374640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e27000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1374640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e27000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51964/0 (socket says 172.21.15.103:51964) 2024-09-05T02:07:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1374640 1 -- 172.21.15.103:0/2918931470 learned_addr learned my addr 172.21.15.103:0/2918931470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1b75640 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 0x5564b8e97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b0b73640 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 0x5564b8e27540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1374640 1 -- 172.21.15.103:0/2918931470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 msgr2=0x5564b8e97600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1374640 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 0x5564b8e97600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1374640 1 -- 172.21.15.103:0/2918931470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 msgr2=0x5564b8e27540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1374640 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 0x5564b8e27540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1374640 1 -- 172.21.15.103:0/2918931470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564b8e97b40 con 0x5564b8e189a0 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.811+0000 7f84b1b75640 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 0x5564b8e97600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.812+0000 7f84b1374640 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e27000 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f84a802f040 tx=0x7f84a80042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f2597013d3749e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.812+0000 7f8494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.812+0000 7f8494ff9640 1 -- 172.21.15.103:0/2918931470 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a803a060 con 0x5564b8e189a0 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.812+0000 7f8494ff9640 1 -- 172.21.15.103:0/2918931470 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f84a80036f0 con 0x5564b8e189a0 2024-09-05T02:07:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.812+0000 7f84b6449740 1 -- 172.21.15.103:0/2918931470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a8004490 con 0x5564b8e189a0 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.812+0000 7f8494ff9640 1 -- 172.21.15.103:0/2918931470 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f84a8041050 con 0x5564b8e189a0 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.812+0000 7f84b6449740 10 client.15990 fetch_fsmap learned FSMap version 35 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.812+0000 7f84b6449740 1 -- 172.21.15.103:0/2918931470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5564b8e98d60 con 0x5564b8e189a0 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.812+0000 7f8494ff9640 1 -- 172.21.15.103:0/2918931470 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a8004490 con 0x5564b8e189a0 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f8494ff9640 1 -- 172.21.15.103:0/2918931470 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f84a800fa50 con 0x5564b8e189a0 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f84b6449740 10 client.15990 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f84b6449740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f84b6449740 20 client.15990 populate_metadata read hostname 'smithi103' 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f84b6449740 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5564b8e994a0 0x5564b8e9b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f84b6449740 4 client.15990 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f84b6449740 1 -- 172.21.15.103:0/2918931470 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564b8d83d70 con 0x5564b8e994a0 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f84b0b73640 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5564b8e994a0 0x5564b8e9b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f84b0b73640 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5564b8e994a0 0x5564b8e9b8c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd5d20238460d7e7 server_cookie=887b39c0022e490a in_seq=0 out_seq=0 2024-09-05T02:07:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.813+0000 7f8494ff9640 10 client.15990 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.872+0000 7f8494ff9640 1 -- 172.21.15.103:0/2918931470 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5564b8d83d70 con 0x5564b8e994a0 2024-09-05T02:07:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f8494ff9640 10 client.15990 handle_command_reply: tid=0 2024-09-05T02:07:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f84b6449740 1 client.15990 shutdown 2024-09-05T02:07:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f84b6449740 1 -- 172.21.15.103:0/2918931470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 msgr2=0x5564b8e27000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f84b6449740 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e27000 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f84a802f040 tx=0x7f84a80042c0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f84b6449740 1 -- 172.21.15.103:0/2918931470 shutdown_connections 2024-09-05T02:07:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f84b6449740 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5564b8e994a0 0x5564b8e9b8c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f84b6449740 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564b8e1e370 0x5564b8e97600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f8494ff9640 0 client.15990 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f84b6449740 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5564b8e192e0 0x5564b8e27540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f84b6449740 1 --2- 172.21.15.103:0/2918931470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564b8e189a0 0x5564b8e27000 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.873+0000 7f84b6449740 1 -- 172.21.15.103:0/2918931470 >> 172.21.15.103:0/2918931470 conn(0x5564b8e14010 msgr2=0x5564b8e131c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.874+0000 7f84b6449740 1 -- 172.21.15.103:0/2918931470 shutdown_connections 2024-09-05T02:07:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.874+0000 7f84b6449740 1 -- 172.21.15.103:0/2918931470 wait complete. 2024-09-05T02:07:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b6449740 20 client.15990 trim_cache size 0 max 16384 2024-09-05T02:07:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:07:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:07:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:07:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:07:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 -- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f84980a2cb0 msgr2=0x7f84980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f84980a2cb0 0x7f84980a5170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f849c009230 tx=0x7f849c000bc0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 -- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac102610 msgr2=0x7f84ac1a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac102610 0x7f84ac1a14d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f84a400eb10 tx=0x7f84a400c2d0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 -- 172.21.15.103:0/846458809 shutdown_connections 2024-09-05T02:07:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84ac102610 0x7f84ac1a14d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f84980a2cb0 0x7f84980a5170 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84ac101c50 0x7f84ac19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 --2- 172.21.15.103:0/846458809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac100a60 0x7f84ac199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.875+0000 7f84b35d8640 1 -- 172.21.15.103:0/846458809 >> 172.21.15.103:0/846458809 conn(0x7f84ac0fc1c0 msgr2=0x7f84ac105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.876+0000 7f84b35d8640 1 -- 172.21.15.103:0/846458809 shutdown_connections 2024-09-05T02:07:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:08.876+0000 7f84b35d8640 1 -- 172.21.15.103:0/846458809 wait complete. 2024-09-05T02:07:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:10 smithi167 ceph-mon[35122]: pgmap v492: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-09-05T02:07:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:10 smithi139 ceph-mon[34262]: pgmap v492: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-09-05T02:07:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:10 smithi103 ceph-mon[30902]: pgmap v492: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-09-05T02:07:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:12 smithi167 ceph-mon[35122]: pgmap v493: 129 pgs: 129 active+clean; 261 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-09-05T02:07:12.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:12 smithi139 ceph-mon[34262]: pgmap v493: 129 pgs: 129 active+clean; 261 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-09-05T02:07:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:12 smithi103 ceph-mon[30902]: pgmap v493: 129 pgs: 129 active+clean; 261 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-09-05T02:07:13.875 INFO:tasks.workunit.client.0.smithi103.stdout: 53 12400 114 7618 78 2955 131 2024-09-05T02:07:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:14 smithi167 ceph-mon[35122]: pgmap v494: 129 pgs: 129 active+clean; 262 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-09-05T02:07:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:14 smithi139 ceph-mon[34262]: pgmap v494: 129 pgs: 129 active+clean; 262 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-09-05T02:07:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:14 smithi103 ceph-mon[30902]: pgmap v494: 129 pgs: 129 active+clean; 262 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-09-05T02:07:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:16 smithi167 ceph-mon[35122]: pgmap v495: 129 pgs: 129 active+clean; 269 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-09-05T02:07:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:16 smithi139 ceph-mon[34262]: pgmap v495: 129 pgs: 129 active+clean; 269 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-09-05T02:07:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:16 smithi103 ceph-mon[30902]: pgmap v495: 129 pgs: 129 active+clean; 269 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-09-05T02:07:17.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:17 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:07:17.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:17 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:07:17.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:17 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:07:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:18 smithi167 ceph-mon[35122]: pgmap v496: 129 pgs: 129 active+clean; 270 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-09-05T02:07:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:07:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:07:18.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:07:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:18 smithi139 ceph-mon[34262]: pgmap v496: 129 pgs: 129 active+clean; 270 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-09-05T02:07:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:07:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:07:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:07:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:18 smithi103 ceph-mon[30902]: pgmap v496: 129 pgs: 129 active+clean; 270 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-09-05T02:07:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:07:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:07:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:07:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:20 smithi167 ceph-mon[35122]: pgmap v497: 129 pgs: 129 active+clean; 274 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-09-05T02:07:20.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:20 smithi139 ceph-mon[34262]: pgmap v497: 129 pgs: 129 active+clean; 274 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-09-05T02:07:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:20 smithi103 ceph-mon[30902]: pgmap v497: 129 pgs: 129 active+clean; 274 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-09-05T02:07:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.997+0000 7fee5614b640 1 -- 172.21.15.103:0/2796547698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee50104f40 msgr2=0x7fee501053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.997+0000 7fee5614b640 1 --2- 172.21.15.103:0/2796547698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee50104f40 0x7fee501053c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee38009860 tx=0x7fee3802a340 comp rx=0 tx=0).stop 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.997+0000 7fee5614b640 1 -- 172.21.15.103:0/2796547698 shutdown_connections 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.997+0000 7fee5614b640 1 --2- 172.21.15.103:0/2796547698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee50105900 0x7fee5010c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.997+0000 7fee5614b640 1 --2- 172.21.15.103:0/2796547698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee50104f40 0x7fee501053c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.997+0000 7fee5614b640 1 --2- 172.21.15.103:0/2796547698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50103d50 0x7fee50104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.998+0000 7fee5614b640 1 -- 172.21.15.103:0/2796547698 >> 172.21.15.103:0/2796547698 conn(0x7fee500ff4b0 msgr2=0x7fee501018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.998+0000 7fee5614b640 1 -- 172.21.15.103:0/2796547698 shutdown_connections 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.998+0000 7fee5614b640 1 -- 172.21.15.103:0/2796547698 wait complete. 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.998+0000 7fee5614b640 1 Processor -- start 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.998+0000 7fee5614b640 1 -- start start 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee5614b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee50103d50 0x7fee50199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee5614b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee50104f40 0x7fee5019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee5614b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50105900 0x7fee501a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee5614b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee5010f860 con 0x7fee50104f40 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee5614b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee5010f6e0 con 0x7fee50103d50 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee5614b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee5010f9e0 con 0x7fee50105900 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee50104f40 0x7fee5019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee50104f40 0x7fee5019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50862/0 (socket says 172.21.15.103:50862) 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee47fff640 1 -- 172.21.15.103:0/1418249470 learned_addr learned my addr 172.21.15.103:0/1418249470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee50103d50 0x7fee50199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee4ffff640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50105900 0x7fee501a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee47fff640 1 -- 172.21.15.103:0/1418249470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50105900 msgr2=0x7fee501a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee47fff640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50105900 0x7fee501a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee47fff640 1 -- 172.21.15.103:0/1418249470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee50103d50 msgr2=0x7fee50199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee47fff640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee50103d50 0x7fee50199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee47fff640 1 -- 172.21.15.103:0/1418249470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee501a1b00 con 0x7fee50104f40 2024-09-05T02:07:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:21.999+0000 7fee47fff640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee50104f40 0x7fee5019a420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fee38005940 tx=0x7fee38030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.000+0000 7fee4d7fa640 1 -- 172.21.15.103:0/1418249470 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38037050 con 0x7fee50104f40 2024-09-05T02:07:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.000+0000 7fee4d7fa640 1 -- 172.21.15.103:0/1418249470 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee38043070 con 0x7fee50104f40 2024-09-05T02:07:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.000+0000 7fee4d7fa640 1 -- 172.21.15.103:0/1418249470 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38003c60 con 0x7fee50104f40 2024-09-05T02:07:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.000+0000 7fee5614b640 1 -- 172.21.15.103:0/1418249470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee501a1d90 con 0x7fee50104f40 2024-09-05T02:07:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.000+0000 7fee5614b640 1 -- 172.21.15.103:0/1418249470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee501a21d0 con 0x7fee50104f40 2024-09-05T02:07:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.003+0000 7fee4d7fa640 1 -- 172.21.15.103:0/1418249470 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fee38004040 con 0x7fee50104f40 2024-09-05T02:07:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee4d7fa640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fee3c0a2cb0 0x7fee3c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee58fc2740 1 Processor -- start 2024-09-05T02:07:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee4d7fa640 1 -- 172.21.15.103:0/1418249470 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fee380ef990 con 0x7fee50104f40 2024-09-05T02:07:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee58fc2740 1 -- start start 2024-09-05T02:07:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fee3c0a2cb0 0x7fee3c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9faf0 0x55f73cfa45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfa7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee58fc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f73cefa2f0 con 0x55f73cfa4b80 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee58fc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f73cea2270 con 0x55f73cf9faf0 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee58fc2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f73cef62b0 con 0x55f73cf9f1b0 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee4f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee4f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:55134/0 (socket says 172.21.15.103:55134) 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee4f7fe640 1 -- 172.21.15.103:0/3961597874 learned_addr learned my addr 172.21.15.103:0/3961597874 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee47fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9faf0 0x55f73cfa45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.004+0000 7fee4ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfa7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fee3c0a2cb0 0x7fee3c0a5170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee40011630 tx=0x7fee40011a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee47fff640 1 -- 172.21.15.103:0/3961597874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9f1b0 msgr2=0x55f73cf9f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee47fff640 1 --2- 172.21.15.103:0/3961597874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee47fff640 1 -- 172.21.15.103:0/3961597874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 msgr2=0x55f73cfa7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee47fff640 1 --2- 172.21.15.103:0/3961597874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfa7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee47fff640 1 -- 172.21.15.103:0/3961597874 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73cea1340 con 0x55f73cf9faf0 2024-09-05T02:07:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/3961597874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee47fff640 1 --2- 172.21.15.103:0/3961597874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9faf0 0x55f73cfa45b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee380ed130 tx=0x7fee38030960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b77b1247040a8f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee45ffb640 1 -- 172.21.15.103:0/3961597874 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee380be070 con 0x55f73cf9faf0 2024-09-05T02:07:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee45ffb640 1 -- 172.21.15.103:0/3961597874 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee38038040 con 0x55f73cf9faf0 2024-09-05T02:07:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.005+0000 7fee45ffb640 1 -- 172.21.15.103:0/3961597874 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee380ba140 con 0x55f73cf9faf0 2024-09-05T02:07:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.006+0000 7fee58fc2740 1 -- 172.21.15.103:0/3961597874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9faf0 msgr2=0x55f73cfa45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.006+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3961597874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9faf0 0x55f73cfa45b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee380ed130 tx=0x7fee38030960 comp rx=0 tx=0).stop 2024-09-05T02:07:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.006+0000 7fee58fc2740 1 -- 172.21.15.103:0/3961597874 shutdown_connections 2024-09-05T02:07:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.006+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3961597874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfa7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.006+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3961597874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9faf0 0x55f73cfa45b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.006+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3961597874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.006+0000 7fee58fc2740 1 -- 172.21.15.103:0/3961597874 >> 172.21.15.103:0/3961597874 conn(0x55f73cf92010 msgr2=0x55f73cf92410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.006+0000 7fee58fc2740 1 -- 172.21.15.103:0/3961597874 shutdown_connections 2024-09-05T02:07:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.006+0000 7fee58fc2740 1 -- 172.21.15.103:0/3961597874 wait complete. 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee58fc2740 1 -- start start 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee58fc2740 1 Processor -- start 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfa8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9faf0 0x55f73cfa94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cfa4b80 0x55f73cfb0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee58fc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f73cea2270 con 0x55f73cf9faf0 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee58fc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f73ce83290 con 0x55f73cf9f1b0 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee58fc2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f73cfa7910 con 0x55f73cfa4b80 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee58fc2740 10 client.0 _ll_register_callbacks cb 0x55f73ceb4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9faf0 0x55f73cfa94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee4ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cfa4b80 0x55f73cfb0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee4f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfa8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9faf0 0x55f73cfa94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50880/0 (socket says 172.21.15.103:50880) 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee4ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cfa4b80 0x55f73cfb0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:55136/0 (socket says 172.21.15.103:55136) 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee47fff640 1 -- 172.21.15.103:0/3879998189 learned_addr learned my addr 172.21.15.103:0/3879998189 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.007+0000 7fee4f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfa8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:39010/0 (socket says 172.21.15.103:39010) 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee4ffff640 1 -- 172.21.15.103:0/3879998189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 msgr2=0x55f73cfa8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee4ffff640 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfa8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee4ffff640 1 -- 172.21.15.103:0/3879998189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9faf0 msgr2=0x55f73cfa94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee4ffff640 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9faf0 0x55f73cfa94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee4ffff640 1 -- 172.21.15.103:0/3879998189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73cc0d210 con 0x55f73cfa4b80 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee47fff640 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9faf0 0x55f73cfa94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfa8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee4ffff640 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cfa4b80 0x55f73cfb0500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fee48005fd0 tx=0x7fee4800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f858143edbd8ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee2effd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:07:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee2effd640 1 -- 172.21.15.103:0/3879998189 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48011e60 con 0x55f73cfa4b80 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee58fc2740 1 -- 172.21.15.103:0/3879998189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee38038040 con 0x55f73cfa4b80 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee2effd640 1 -- 172.21.15.103:0/3879998189 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fee48004b20 con 0x55f73cfa4b80 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee2effd640 1 -- 172.21.15.103:0/3879998189 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fee4800fe60 con 0x55f73cfa4b80 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee58fc2740 10 client.25366 fetch_fsmap learned FSMap version 35 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.008+0000 7fee58fc2740 1 -- 172.21.15.103:0/3879998189 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f73ccd9620 con 0x55f73cfa4b80 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee2effd640 1 -- 172.21.15.103:0/3879998189 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee38038040 con 0x55f73cfa4b80 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee2effd640 1 -- 172.21.15.103:0/3879998189 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fee48019070 con 0x55f73cfa4b80 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee58fc2740 10 client.25366 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee58fc2740 10 client.25366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee58fc2740 20 client.25366 populate_metadata read hostname 'smithi103' 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f73cfb5100 0x55f73cfb7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee58fc2740 4 client.25366 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee58fc2740 1 -- 172.21.15.103:0/3879998189 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f73cf07a20 con 0x55f73cfb5100 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f73cfb5100 0x55f73cfb7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee2effd640 10 client.25366 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.009+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f73cfb5100 0x55f73cfb7540 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc2fda3622a6921 server_cookie=2a000f4d2bd981a3 in_seq=0 out_seq=0 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.010+0000 7fee2effd640 1 -- 172.21.15.103:0/3879998189 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f73cf07a20 con 0x55f73cfb5100 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.010+0000 7fee2effd640 10 client.25366 handle_command_reply: tid=0 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.010+0000 7fee58fc2740 1 client.25366 shutdown 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.010+0000 7fee58fc2740 1 -- 172.21.15.103:0/3879998189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cfa4b80 msgr2=0x55f73cfb0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.010+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cfa4b80 0x55f73cfb0500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fee48005fd0 tx=0x7fee4800f790 comp rx=0 tx=0).stop 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.010+0000 7fee58fc2740 1 -- 172.21.15.103:0/3879998189 shutdown_connections 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.010+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f73cfb5100 0x55f73cfb7540 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.011+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cfa4b80 0x55f73cfb0500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.011+0000 7fee2effd640 0 client.25366 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.011+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9faf0 0x55f73cfa94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.011+0000 7fee58fc2740 1 --2- 172.21.15.103:0/3879998189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfa8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.011+0000 7fee58fc2740 1 -- 172.21.15.103:0/3879998189 >> 172.21.15.103:0/3879998189 conn(0x55f73cf92010 msgr2=0x55f73cf93820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.011+0000 7fee58fc2740 1 -- 172.21.15.103:0/3879998189 shutdown_connections 2024-09-05T02:07:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.011+0000 7fee58fc2740 1 -- 172.21.15.103:0/3879998189 wait complete. 2024-09-05T02:07:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.011+0000 7fee58fc2740 20 client.25366 trim_cache size 0 max 16384 2024-09-05T02:07:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.038+0000 7fee58fc2740 1 Processor -- start 2024-09-05T02:07:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.038+0000 7fee58fc2740 1 -- start start 2024-09-05T02:07:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.038+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.038+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfb04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.038+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cfa4b80 0x55f73cf923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.038+0000 7fee58fc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee501a84d0 con 0x55f73cf9f1b0 2024-09-05T02:07:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.038+0000 7fee58fc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee501aa0d0 con 0x55f73cfa4b80 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.038+0000 7fee58fc2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee501a8ba0 con 0x55f73cf9faf0 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee4f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee47fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfb04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee4ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cfa4b80 0x55f73cf923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee47fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfb04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:55148/0 (socket says 172.21.15.103:55148) 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee47fff640 1 -- 172.21.15.103:0/4132924150 learned_addr learned my addr 172.21.15.103:0/4132924150 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee4f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50890/0 (socket says 172.21.15.103:50890) 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee47fff640 1 -- 172.21.15.103:0/4132924150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cfa4b80 msgr2=0x55f73cf923f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee47fff640 1 --2- 172.21.15.103:0/4132924150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cfa4b80 0x55f73cf923f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee47fff640 1 -- 172.21.15.103:0/4132924150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9f1b0 msgr2=0x55f73cf9f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee47fff640 1 --2- 172.21.15.103:0/4132924150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee47fff640 1 -- 172.21.15.103:0/4132924150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73cfa0780 con 0x55f73cf9faf0 2024-09-05T02:07:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/4132924150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee47fff640 1 --2- 172.21.15.103:0/4132924150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfb04c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee38009500 tx=0x7fee380c01f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fbd8407a75ab1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee457fa640 1 -- 172.21.15.103:0/4132924150 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee380be030 con 0x55f73cf9faf0 2024-09-05T02:07:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee457fa640 1 -- 172.21.15.103:0/4132924150 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee380bc540 con 0x55f73cf9faf0 2024-09-05T02:07:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.039+0000 7fee457fa640 1 -- 172.21.15.103:0/4132924150 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee380ba140 con 0x55f73cf9faf0 2024-09-05T02:07:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 -- 172.21.15.103:0/4132924150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 msgr2=0x55f73cfb04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 --2- 172.21.15.103:0/4132924150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfb04c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee38009500 tx=0x7fee380c01f0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 -- 172.21.15.103:0/4132924150 shutdown_connections 2024-09-05T02:07:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 --2- 172.21.15.103:0/4132924150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cfa4b80 0x55f73cf923f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 --2- 172.21.15.103:0/4132924150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfb04c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 --2- 172.21.15.103:0/4132924150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cf9f1b0 0x55f73cf9f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 -- 172.21.15.103:0/4132924150 >> 172.21.15.103:0/4132924150 conn(0x55f73cf92010 msgr2=0x55f73cfa4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 -- 172.21.15.103:0/4132924150 shutdown_connections 2024-09-05T02:07:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 -- 172.21.15.103:0/4132924150 wait complete. 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 -- start start 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.041+0000 7fee58fc2740 1 Processor -- start 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfb00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfacb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee58fc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee58fc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee501a9a00 con 0x55f73cfa4b80 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee58fc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee501a9270 con 0x55f73cf9f1b0 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee58fc2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee501a64b0 con 0x55f73cf9faf0 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee58fc2740 10 client.0 _ll_register_callbacks cb 0x7fee48009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50898/0 (socket says 172.21.15.103:50898) 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4ffff640 1 -- 172.21.15.103:0/1886991055 learned_addr learned my addr 172.21.15.103:0/1886991055 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee47fff640 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfacb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfb00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4ffff640 1 -- 172.21.15.103:0/1886991055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 msgr2=0x55f73cfacb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4ffff640 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfacb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4ffff640 1 -- 172.21.15.103:0/1886991055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 msgr2=0x55f73cfb00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4ffff640 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfb00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4ffff640 1 -- 172.21.15.103:0/1886991055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73cfb0750 con 0x55f73cfa4b80 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.042+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfb00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee4ffff640 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfad0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee48005fd0 tx=0x7fee48005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1b7e57ef9cf25e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee2effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee2effd640 1 -- 172.21.15.103:0/1886991055 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48005490 con 0x55f73cfa4b80 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee2effd640 1 -- 172.21.15.103:0/1886991055 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fee48005630 con 0x55f73cfa4b80 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee58fc2740 1 -- 172.21.15.103:0/1886991055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee380bc540 con 0x55f73cfa4b80 2024-09-05T02:07:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee2effd640 1 -- 172.21.15.103:0/1886991055 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fee4803a050 con 0x55f73cfa4b80 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee58fc2740 10 client.16008 fetch_fsmap learned FSMap version 35 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee58fc2740 1 -- 172.21.15.103:0/1886991055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f73d01ed50 con 0x55f73cfa4b80 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee2effd640 1 -- 172.21.15.103:0/1886991055 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee380bc540 con 0x55f73cfa4b80 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.043+0000 7fee2effd640 1 -- 172.21.15.103:0/1886991055 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fee48019040 con 0x55f73cfa4b80 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.044+0000 7fee58fc2740 10 client.16008 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.044+0000 7fee58fc2740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.044+0000 7fee58fc2740 20 client.16008 populate_metadata read hostname 'smithi103' 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.044+0000 7fee58fc2740 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f73d01f730 0x55f73d021b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.044+0000 7fee58fc2740 4 client.16008 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.044+0000 7fee58fc2740 1 -- 172.21.15.103:0/1886991055 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f73cf01d70 con 0x55f73d01f730 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.044+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f73d01f730 0x55f73d021b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.044+0000 7fee2effd640 10 client.16008 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.044+0000 7fee4f7fe640 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f73d01f730 0x55f73d021b50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdded8430b183007 server_cookie=1b0a01666f9c3884 in_seq=0 out_seq=0 2024-09-05T02:07:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee2effd640 1 -- 172.21.15.103:0/1886991055 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f73cf01d70 con 0x55f73d01f730 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee2effd640 10 client.16008 handle_command_reply: tid=0 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee58fc2740 1 client.16008 shutdown 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee58fc2740 1 -- 172.21.15.103:0/1886991055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 msgr2=0x55f73cfad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee58fc2740 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfad0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee48005fd0 tx=0x7fee48005190 comp rx=0 tx=0).stop 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee58fc2740 1 -- 172.21.15.103:0/1886991055 shutdown_connections 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee58fc2740 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f73d01f730 0x55f73d021b50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee58fc2740 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73cfa4b80 0x55f73cfad0d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee58fc2740 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f73cf9faf0 0x55f73cfacb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee58fc2740 1 --2- 172.21.15.103:0/1886991055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f73cf9f1b0 0x55f73cfb00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee58fc2740 1 -- 172.21.15.103:0/1886991055 >> 172.21.15.103:0/1886991055 conn(0x55f73cf92010 msgr2=0x55f73cf909a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.172+0000 7fee2effd640 0 client.16008 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.173+0000 7fee58fc2740 1 -- 172.21.15.103:0/1886991055 shutdown_connections 2024-09-05T02:07:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.173+0000 7fee58fc2740 1 -- 172.21.15.103:0/1886991055 wait complete. 2024-09-05T02:07:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.173+0000 7fee58fc2740 20 client.16008 trim_cache size 0 max 16384 2024-09-05T02:07:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:07:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:07:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:07:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:07:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 -- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fee3c0a2cb0 msgr2=0x7fee3c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fee3c0a2cb0 0x7fee3c0a5170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee40011630 tx=0x7fee40011a30 comp rx=0 tx=0).stop 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 -- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee50104f40 msgr2=0x7fee5019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee50104f40 0x7fee5019a420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fee38005940 tx=0x7fee38030e00 comp rx=0 tx=0).stop 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 -- 172.21.15.103:0/1418249470 shutdown_connections 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fee3c0a2cb0 0x7fee3c0a5170 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50105900 0x7fee501a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee50104f40 0x7fee5019a420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 --2- 172.21.15.103:0/1418249470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee50103d50 0x7fee50199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 -- 172.21.15.103:0/1418249470 >> 172.21.15.103:0/1418249470 conn(0x7fee500ff4b0 msgr2=0x7fee501082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 -- 172.21.15.103:0/1418249470 shutdown_connections 2024-09-05T02:07:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:22.174+0000 7fee5614b640 1 -- 172.21.15.103:0/1418249470 wait complete. 2024-09-05T02:07:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:22 smithi167 ceph-mon[35122]: pgmap v498: 129 pgs: 129 active+clean; 279 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-09-05T02:07:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:22 smithi139 ceph-mon[34262]: pgmap v498: 129 pgs: 129 active+clean; 279 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-09-05T02:07:22.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:22 smithi103 ceph-mon[30902]: pgmap v498: 129 pgs: 129 active+clean; 279 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-09-05T02:07:23.875 INFO:tasks.workunit.client.0.smithi103.stdout: 54 12690 149 7341 98 3365 127 2024-09-05T02:07:24.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:24 smithi167 ceph-mon[35122]: pgmap v499: 129 pgs: 129 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-09-05T02:07:24.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:24 smithi139 ceph-mon[34262]: pgmap v499: 129 pgs: 129 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-09-05T02:07:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:24 smithi103 ceph-mon[30902]: pgmap v499: 129 pgs: 129 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-09-05T02:07:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:26 smithi167 ceph-mon[35122]: pgmap v500: 129 pgs: 129 active+clean; 286 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-09-05T02:07:26.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:26 smithi139 ceph-mon[34262]: pgmap v500: 129 pgs: 129 active+clean; 286 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-09-05T02:07:26.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:26 smithi103 ceph-mon[30902]: pgmap v500: 129 pgs: 129 active+clean; 286 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-09-05T02:07:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:28 smithi167 ceph-mon[35122]: pgmap v501: 129 pgs: 129 active+clean; 289 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-09-05T02:07:28.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:28 smithi139 ceph-mon[34262]: pgmap v501: 129 pgs: 129 active+clean; 289 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-09-05T02:07:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:28 smithi103 ceph-mon[30902]: pgmap v501: 129 pgs: 129 active+clean; 289 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-09-05T02:07:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:30 smithi167 ceph-mon[35122]: pgmap v502: 129 pgs: 129 active+clean; 291 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-09-05T02:07:30.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:30 smithi139 ceph-mon[34262]: pgmap v502: 129 pgs: 129 active+clean; 291 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-09-05T02:07:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:30 smithi103 ceph-mon[30902]: pgmap v502: 129 pgs: 129 active+clean; 291 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-09-05T02:07:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:32 smithi167 ceph-mon[35122]: pgmap v503: 129 pgs: 129 active+clean; 295 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-09-05T02:07:32.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:32 smithi139 ceph-mon[34262]: pgmap v503: 129 pgs: 129 active+clean; 295 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-09-05T02:07:32.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:32 smithi103 ceph-mon[30902]: pgmap v503: 129 pgs: 129 active+clean; 295 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-09-05T02:07:33.875 INFO:tasks.workunit.client.0.smithi103.stdout: 55 12260 85 7200 88 3896 108 2024-09-05T02:07:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:34 smithi167 ceph-mon[35122]: pgmap v504: 129 pgs: 129 active+clean; 298 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-09-05T02:07:34.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:34 smithi139 ceph-mon[34262]: pgmap v504: 129 pgs: 129 active+clean; 298 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-09-05T02:07:34.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:34 smithi103 ceph-mon[30902]: pgmap v504: 129 pgs: 129 active+clean; 298 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-09-05T02:07:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.298+0000 7fe103074640 1 -- 172.21.15.103:0/2464373433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0fc103710 msgr2=0x7fe0fc109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.298+0000 7fe103074640 1 --2- 172.21.15.103:0/2464373433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0fc103710 0x7fe0fc109fa0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009ca0 tx=0x7fe0f402a8c0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.299+0000 7fe103074640 1 -- 172.21.15.103:0/2464373433 shutdown_connections 2024-09-05T02:07:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.299+0000 7fe103074640 1 --2- 172.21.15.103:0/2464373433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0fc103710 0x7fe0fc109fa0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.299+0000 7fe103074640 1 --2- 172.21.15.103:0/2464373433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc102d50 0x7fe0fc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.299+0000 7fe103074640 1 --2- 172.21.15.103:0/2464373433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0fc101b60 0x7fe0fc101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.299+0000 7fe103074640 1 -- 172.21.15.103:0/2464373433 >> 172.21.15.103:0/2464373433 conn(0x7fe0fc0fd2e0 msgr2=0x7fe0fc0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.299+0000 7fe103074640 1 -- 172.21.15.103:0/2464373433 shutdown_connections 2024-09-05T02:07:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.299+0000 7fe103074640 1 -- 172.21.15.103:0/2464373433 wait complete. 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.299+0000 7fe103074640 1 Processor -- start 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.299+0000 7fe103074640 1 -- start start 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe103074640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc101b60 0x7fe0fc075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe103074640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0fc102d50 0x7fe0fc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe103074640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0fc103710 0x7fe0fc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe103074640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc10d670 con 0x7fe0fc103710 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe103074640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc10d4f0 con 0x7fe0fc102d50 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe103074640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc10d7f0 con 0x7fe0fc101b60 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe101611640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0fc103710 0x7fe0fc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe0fbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0fc102d50 0x7fe0fc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe0fbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0fc102d50 0x7fe0fc0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:40050/0 (socket says 172.21.15.103:40050) 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe0fbfff640 1 -- 172.21.15.103:0/2828693803 learned_addr learned my addr 172.21.15.103:0/2828693803 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe100e10640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc101b60 0x7fe0fc075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe0fbfff640 1 -- 172.21.15.103:0/2828693803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc101b60 msgr2=0x7fe0fc075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc101b60 0x7fe0fc075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe0fbfff640 1 -- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0fc103710 msgr2=0x7fe0fc074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0fc103710 0x7fe0fc074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe0fbfff640 1 -- 172.21.15.103:0/2828693803 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc0753b0 con 0x7fe0fc102d50 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe101611640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0fc103710 0x7fe0fc074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.300+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0fc102d50 0x7fe0fc0745e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800e9f0 tx=0x7fe0e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.301+0000 7fe0f9ffb640 1 -- 172.21.15.103:0/2828693803 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800cd80 con 0x7fe0fc102d50 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.301+0000 7fe103074640 1 -- 172.21.15.103:0/2828693803 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc1a1d80 con 0x7fe0fc102d50 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.301+0000 7fe0f9ffb640 1 -- 172.21.15.103:0/2828693803 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0e80108e0 con 0x7fe0fc102d50 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.301+0000 7fe0f9ffb640 1 -- 172.21.15.103:0/2828693803 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8018920 con 0x7fe0fc102d50 2024-09-05T02:07:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.301+0000 7fe103074640 1 -- 172.21.15.103:0/2828693803 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc1a2240 con 0x7fe0fc102d50 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.304+0000 7fe0f9ffb640 1 -- 172.21.15.103:0/2828693803 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe0e8010a90 con 0x7fe0fc102d50 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.304+0000 7fe105e15740 1 Processor -- start 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.304+0000 7fe105e15740 1 -- start start 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.304+0000 7fe105e15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e509a0 0x55f3f5e50da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.304+0000 7fe105e15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.304+0000 7fe105e15740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e58830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.304+0000 7fe105e15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3f5db42f0 con 0x55f3f5e509a0 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.304+0000 7fe105e15740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f3f5d5c270 con 0x55f3f5e56370 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.304+0000 7fe105e15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3f5db02b0 con 0x55f3f5e512e0 2024-09-05T02:07:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe101611640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e509a0 0x55f3f5e50da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe100e10640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e55da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe101611640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e509a0 0x55f3f5e50da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43394/0 (socket says 172.21.15.103:43394) 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe101611640 1 -- 172.21.15.103:0/823957233 learned_addr learned my addr 172.21.15.103:0/823957233 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/823957233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e58830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe100e10640 1 -- 172.21.15.103:0/823957233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 msgr2=0x55f3f5e58830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe100e10640 1 --2- 172.21.15.103:0/823957233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e58830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe100e10640 1 -- 172.21.15.103:0/823957233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e509a0 msgr2=0x55f3f5e50da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe100e10640 1 --2- 172.21.15.103:0/823957233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e509a0 0x55f3f5e50da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe100e10640 1 -- 172.21.15.103:0/823957233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f5d5b340 con 0x55f3f5e512e0 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/823957233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e58830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe0f9ffb640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0ec0a2cb0 0x7fe0ec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe100e10640 1 --2- 172.21.15.103:0/823957233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e55da0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc102bb0 tx=0x7fe0f000bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4157ab48bffee614 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe101611640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0ec0a2cb0 0x7fe0ec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.305+0000 7fe0f9ffb640 1 -- 172.21.15.103:0/2828693803 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe0e80cf620 con 0x7fe0fc102d50 2024-09-05T02:07:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.306+0000 7fe0e6ffd640 1 -- 172.21.15.103:0/823957233 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0003dd0 con 0x55f3f5e512e0 2024-09-05T02:07:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.306+0000 7fe0e6ffd640 1 -- 172.21.15.103:0/823957233 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f0015070 con 0x55f3f5e512e0 2024-09-05T02:07:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.306+0000 7fe0e6ffd640 1 -- 172.21.15.103:0/823957233 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00116a0 con 0x55f3f5e512e0 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.306+0000 7fe105e15740 1 -- 172.21.15.103:0/823957233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 msgr2=0x55f3f5e55da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.306+0000 7fe105e15740 1 --2- 172.21.15.103:0/823957233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e55da0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc102bb0 tx=0x7fe0f000bbe0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.306+0000 7fe101611640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0ec0a2cb0 0x7fe0ec0a5170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4005e30 tx=0x7fe0f4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.307+0000 7fe105e15740 1 -- 172.21.15.103:0/823957233 shutdown_connections 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.307+0000 7fe105e15740 1 --2- 172.21.15.103:0/823957233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e58830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.307+0000 7fe105e15740 1 --2- 172.21.15.103:0/823957233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e55da0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.307+0000 7fe105e15740 1 --2- 172.21.15.103:0/823957233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e509a0 0x55f3f5e50da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.307+0000 7fe105e15740 1 -- 172.21.15.103:0/823957233 >> 172.21.15.103:0/823957233 conn(0x55f3f5e4c010 msgr2=0x55f3f5e4c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.307+0000 7fe105e15740 1 -- 172.21.15.103:0/823957233 shutdown_connections 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.307+0000 7fe105e15740 1 -- 172.21.15.103:0/823957233 wait complete. 2024-09-05T02:07:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.307+0000 7fe105e15740 1 -- start start 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.307+0000 7fe105e15740 1 Processor -- start 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe105e15740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e5acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe105e15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e5b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe105e15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 0x55f3f5e62230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe105e15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3f5e59340 con 0x55f3f5e56370 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe105e15740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f3f5d3d290 con 0x55f3f5e509a0 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe105e15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3f5e594c0 con 0x55f3f5e512e0 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe100e10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e5acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe100e10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e5acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:40066/0 (socket says 172.21.15.103:40066) 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe0fbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e5b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe100e10640 1 -- 172.21.15.103:0/3683890445 learned_addr learned my addr 172.21.15.103:0/3683890445 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe101611640 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 0x55f3f5e62230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe105e15740 10 client.0 _ll_register_callbacks cb 0x55f3f5d6e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe0fbfff640 1 -- 172.21.15.103:0/3683890445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 msgr2=0x55f3f5e5acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e5acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe0fbfff640 1 -- 172.21.15.103:0/3683890445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 msgr2=0x55f3f5e62230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 0x55f3f5e62230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.308+0000 7fe0fbfff640 1 -- 172.21.15.103:0/3683890445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3f5ac7210 con 0x55f3f5e512e0 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.309+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e5b1f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c6a0 tx=0x7fe0e809b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d096f8480cb40343 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.309+0000 7fe0dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.309+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3683890445 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80a46e0 con 0x55f3f5e512e0 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.309+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3683890445 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe0e80cf310 con 0x55f3f5e512e0 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.309+0000 7fe105e15740 1 -- 172.21.15.103:0/3683890445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f0015070 con 0x55f3f5e512e0 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.309+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3683890445 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe0e80a4250 con 0x55f3f5e512e0 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.309+0000 7fe105e15740 10 client.25384 fetch_fsmap learned FSMap version 35 2024-09-05T02:07:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.309+0000 7fe105e15740 1 -- 172.21.15.103:0/3683890445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f3f5b93620 con 0x55f3f5e512e0 2024-09-05T02:07:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.310+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3683890445 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f0015070 con 0x55f3f5e512e0 2024-09-05T02:07:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.310+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3683890445 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fe0e8097870 con 0x55f3f5e512e0 2024-09-05T02:07:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.310+0000 7fe105e15740 10 client.25384 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:07:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.310+0000 7fe105e15740 10 client.25384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:07:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.310+0000 7fe105e15740 20 client.25384 populate_metadata read hostname 'smithi103' 2024-09-05T02:07:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.310+0000 7fe105e15740 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f3f5e66820 0x55f3f5e68c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.310+0000 7fe105e15740 4 client.25384 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:07:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.310+0000 7fe105e15740 1 -- 172.21.15.103:0/3683890445 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3f5dc1a20 con 0x55f3f5e66820 2024-09-05T02:07:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.310+0000 7fe100e10640 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f3f5e66820 0x55f3f5e68c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.311+0000 7fe100e10640 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f3f5e66820 0x55f3f5e68c90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c68f0748962d69f server_cookie=597bb22aa9eee3d1 in_seq=0 out_seq=0 2024-09-05T02:07:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.311+0000 7fe0dbfff640 10 client.25384 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.311+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3683890445 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3f5dc1a20 con 0x55f3f5e66820 2024-09-05T02:07:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.311+0000 7fe0dbfff640 10 client.25384 handle_command_reply: tid=0 2024-09-05T02:07:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 client.25384 shutdown 2024-09-05T02:07:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 -- 172.21.15.103:0/3683890445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 msgr2=0x55f3f5e5b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e5b1f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c6a0 tx=0x7fe0e809b540 comp rx=0 tx=0).stop 2024-09-05T02:07:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 -- 172.21.15.103:0/3683890445 shutdown_connections 2024-09-05T02:07:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f3f5e66820 0x55f3f5e68c90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 0x55f3f5e62230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e5b1f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 --2- 172.21.15.103:0/3683890445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e5acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe0dbfff640 0 client.25384 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 -- 172.21.15.103:0/3683890445 >> 172.21.15.103:0/3683890445 conn(0x55f3f5e4c010 msgr2=0x55f3f5e4d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 -- 172.21.15.103:0/3683890445 shutdown_connections 2024-09-05T02:07:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 1 -- 172.21.15.103:0/3683890445 wait complete. 2024-09-05T02:07:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.312+0000 7fe105e15740 20 client.25384 trim_cache size 0 max 16384 2024-09-05T02:07:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe105e15740 1 Processor -- start 2024-09-05T02:07:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe105e15740 1 -- start start 2024-09-05T02:07:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe105e15740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e4d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe105e15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe105e15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 0x55f3f5e4d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe105e15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1a8510 con 0x55f3f5e56370 2024-09-05T02:07:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe105e15740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1aa110 con 0x55f3f5e509a0 2024-09-05T02:07:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe105e15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1a8be0 con 0x55f3f5e512e0 2024-09-05T02:07:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe101611640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 0x55f3f5e4d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.332+0000 7fe100e10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e4d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe101611640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 0x55f3f5e4d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43408/0 (socket says 172.21.15.103:43408) 2024-09-05T02:07:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe101611640 1 -- 172.21.15.103:0/2869450636 learned_addr learned my addr 172.21.15.103:0/2869450636 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/2869450636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe100e10640 1 -- 172.21.15.103:0/2869450636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 msgr2=0x55f3f5e621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe100e10640 1 --2- 172.21.15.103:0/2869450636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe100e10640 1 -- 172.21.15.103:0/2869450636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 msgr2=0x55f3f5e4d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe100e10640 1 --2- 172.21.15.103:0/2869450636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 0x55f3f5e4d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe100e10640 1 -- 172.21.15.103:0/2869450636 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f5e58760 con 0x55f3f5e509a0 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe100e10640 1 --2- 172.21.15.103:0/2869450636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e4d400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0006010 tx=0x7fe0f0013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd47f5c32993366c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe0e67fc640 1 -- 172.21.15.103:0/2869450636 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0003dd0 con 0x55f3f5e509a0 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe0e67fc640 1 -- 172.21.15.103:0/2869450636 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f0015050 con 0x55f3f5e509a0 2024-09-05T02:07:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.333+0000 7fe0e67fc640 1 -- 172.21.15.103:0/2869450636 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00119f0 con 0x55f3f5e509a0 2024-09-05T02:07:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.334+0000 7fe105e15740 1 -- 172.21.15.103:0/2869450636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 msgr2=0x55f3f5e4d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.334+0000 7fe105e15740 1 --2- 172.21.15.103:0/2869450636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e4d400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0006010 tx=0x7fe0f0013040 comp rx=0 tx=0).stop 2024-09-05T02:07:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.335+0000 7fe105e15740 1 -- 172.21.15.103:0/2869450636 shutdown_connections 2024-09-05T02:07:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.335+0000 7fe105e15740 1 --2- 172.21.15.103:0/2869450636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e56370 0x55f3f5e4d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.335+0000 7fe105e15740 1 --2- 172.21.15.103:0/2869450636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e512e0 0x55f3f5e621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.335+0000 7fe105e15740 1 --2- 172.21.15.103:0/2869450636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e509a0 0x55f3f5e4d400 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.335+0000 7fe105e15740 1 -- 172.21.15.103:0/2869450636 >> 172.21.15.103:0/2869450636 conn(0x55f3f5e4c010 msgr2=0x55f3f5e4b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.335+0000 7fe105e15740 1 -- 172.21.15.103:0/2869450636 shutdown_connections 2024-09-05T02:07:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.335+0000 7fe105e15740 1 -- 172.21.15.103:0/2869450636 wait complete. 2024-09-05T02:07:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.335+0000 7fe105e15740 1 -- start start 2024-09-05T02:07:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.335+0000 7fe105e15740 1 Processor -- start 2024-09-05T02:07:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe105e15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e509a0 0x55f3f5e60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe105e15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e512e0 0x55f3f5e60680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe105e15740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e61690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe105e15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1a9a40 con 0x55f3f5e512e0 2024-09-05T02:07:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe105e15740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1a92b0 con 0x55f3f5e56370 2024-09-05T02:07:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe105e15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1a64f0 con 0x55f3f5e509a0 2024-09-05T02:07:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe105e15740 10 client.0 _ll_register_callbacks cb 0x7fe0e8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe0fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e512e0 0x55f3f5e60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe100e10640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e509a0 0x55f3f5e60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe100e10640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e509a0 0x55f3f5e60140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:46702/0 (socket says 172.21.15.103:46702) 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe100e10640 1 -- 172.21.15.103:0/1774492295 learned_addr learned my addr 172.21.15.103:0/1774492295 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe101611640 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e61690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe101611640 1 -- 172.21.15.103:0/1774492295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e509a0 msgr2=0x55f3f5e60140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe101611640 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e509a0 0x55f3f5e60140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe101611640 1 -- 172.21.15.103:0/1774492295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e512e0 msgr2=0x55f3f5e60680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe101611640 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e512e0 0x55f3f5e60680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe101611640 1 -- 172.21.15.103:0/1774492295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3f5e61ca0 con 0x55f3f5e56370 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.336+0000 7fe0fbfff640 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e512e0 0x55f3f5e60680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.337+0000 7fe101611640 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e61690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f3f5e68f50 tx=0x7fe0f40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9e38f150e02dd1e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.337+0000 7fe0dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.338+0000 7fe0dbfff640 1 -- 172.21.15.103:0/1774492295 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4060060 con 0x55f3f5e56370 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.338+0000 7fe0dbfff640 1 -- 172.21.15.103:0/1774492295 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe0f4004720 con 0x55f3f5e56370 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.338+0000 7fe105e15740 1 -- 172.21.15.103:0/1774492295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f0015050 con 0x55f3f5e56370 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe0dbfff640 1 -- 172.21.15.103:0/1774492295 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe0f4067050 con 0x55f3f5e56370 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe105e15740 10 client.25451 fetch_fsmap learned FSMap version 35 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe105e15740 1 -- 172.21.15.103:0/1774492295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f3f5ed1e10 con 0x55f3f5e56370 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe0dbfff640 1 -- 172.21.15.103:0/1774492295 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f0015050 con 0x55f3f5e56370 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe0dbfff640 1 -- 172.21.15.103:0/1774492295 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fe0f402f5e0 con 0x55f3f5e56370 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe105e15740 10 client.25451 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe105e15740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe105e15740 20 client.25451 populate_metadata read hostname 'smithi103' 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe105e15740 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f3f5ed27f0 0x55f3f5ed4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe105e15740 4 client.25451 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:07:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe105e15740 1 -- 172.21.15.103:0/1774492295 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3f5dbbd70 con 0x55f3f5ed27f0 2024-09-05T02:07:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.339+0000 7fe100e10640 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f3f5ed27f0 0x55f3f5ed4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.340+0000 7fe0dbfff640 10 client.25451 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.340+0000 7fe100e10640 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f3f5ed27f0 0x55f3f5ed4c10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372710416e9cf34e server_cookie=7758717241a02543 in_seq=0 out_seq=0 2024-09-05T02:07:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe0dbfff640 1 -- 172.21.15.103:0/1774492295 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3f5dbbd70 con 0x55f3f5ed27f0 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe0dbfff640 10 client.25451 handle_command_reply: tid=0 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe105e15740 1 client.25451 shutdown 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe105e15740 1 -- 172.21.15.103:0/1774492295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 msgr2=0x55f3f5e61690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe105e15740 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e61690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f3f5e68f50 tx=0x7fe0f40095c0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe105e15740 1 -- 172.21.15.103:0/1774492295 shutdown_connections 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe105e15740 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f3f5ed27f0 0x55f3f5ed4c10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe105e15740 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3f5e56370 0x55f3f5e61690 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe105e15740 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3f5e512e0 0x55f3f5e60680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe105e15740 1 --2- 172.21.15.103:0/1774492295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3f5e509a0 0x55f3f5e60140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe0dbfff640 0 client.25451 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.366+0000 7fe105e15740 1 -- 172.21.15.103:0/1774492295 >> 172.21.15.103:0/1774492295 conn(0x55f3f5e4c010 msgr2=0x55f3f5e4c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.367+0000 7fe105e15740 1 -- 172.21.15.103:0/1774492295 shutdown_connections 2024-09-05T02:07:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.367+0000 7fe105e15740 1 -- 172.21.15.103:0/1774492295 wait complete. 2024-09-05T02:07:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.367+0000 7fe105e15740 20 client.25451 trim_cache size 0 max 16384 2024-09-05T02:07:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:07:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:07:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:07:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:07:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 -- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0ec0a2cb0 msgr2=0x7fe0ec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0ec0a2cb0 0x7fe0ec0a5170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4005e30 tx=0x7fe0f4005d80 comp rx=0 tx=0).stop 2024-09-05T02:07:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 -- 172.21.15.103:0/2828693803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0fc102d50 msgr2=0x7fe0fc0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0fc102d50 0x7fe0fc0745e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800e9f0 tx=0x7fe0e800eeb0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 -- 172.21.15.103:0/2828693803 shutdown_connections 2024-09-05T02:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0fc103710 0x7fe0fc074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0fc102d50 0x7fe0fc0745e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe0ec0a2cb0 0x7fe0ec0a5170 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 --2- 172.21.15.103:0/2828693803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc101b60 0x7fe0fc075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 -- 172.21.15.103:0/2828693803 >> 172.21.15.103:0/2828693803 conn(0x7fe0fc0fd2e0 msgr2=0x7fe0fc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.368+0000 7fe103074640 1 -- 172.21.15.103:0/2828693803 shutdown_connections 2024-09-05T02:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:35.369+0000 7fe103074640 1 -- 172.21.15.103:0/2828693803 wait complete. 2024-09-05T02:07:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:36 smithi167 ceph-mon[35122]: pgmap v505: 129 pgs: 129 active+clean; 301 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 227 op/s 2024-09-05T02:07:36.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:36 smithi139 ceph-mon[34262]: pgmap v505: 129 pgs: 129 active+clean; 301 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 227 op/s 2024-09-05T02:07:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:36 smithi103 ceph-mon[30902]: pgmap v505: 129 pgs: 129 active+clean; 301 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 227 op/s 2024-09-05T02:07:37.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:37 smithi139 ceph-mon[34262]: pgmap v506: 129 pgs: 129 active+clean; 301 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-09-05T02:07:37.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:37 smithi103 ceph-mon[30902]: pgmap v506: 129 pgs: 129 active+clean; 301 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-09-05T02:07:37.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:37 smithi167 ceph-mon[35122]: pgmap v506: 129 pgs: 129 active+clean; 301 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-09-05T02:07:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:40 smithi167 ceph-mon[35122]: pgmap v507: 129 pgs: 129 active+clean; 303 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-09-05T02:07:40.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:40 smithi139 ceph-mon[34262]: pgmap v507: 129 pgs: 129 active+clean; 303 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-09-05T02:07:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:40 smithi103 ceph-mon[30902]: pgmap v507: 129 pgs: 129 active+clean; 303 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-09-05T02:07:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:42 smithi167 ceph-mon[35122]: pgmap v508: 129 pgs: 129 active+clean; 304 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-09-05T02:07:42.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:42 smithi139 ceph-mon[34262]: pgmap v508: 129 pgs: 129 active+clean; 304 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-09-05T02:07:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:42 smithi103 ceph-mon[30902]: pgmap v508: 129 pgs: 129 active+clean; 304 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-09-05T02:07:43.875 INFO:tasks.workunit.client.0.smithi103.stdout: 56 12963 62 7668 64 3808 94 2024-09-05T02:07:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:44 smithi167 ceph-mon[35122]: pgmap v509: 129 pgs: 129 active+clean; 307 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-09-05T02:07:44.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:44 smithi139 ceph-mon[34262]: pgmap v509: 129 pgs: 129 active+clean; 307 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-09-05T02:07:44.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:44 smithi103 ceph-mon[30902]: pgmap v509: 129 pgs: 129 active+clean; 307 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-09-05T02:07:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:46 smithi167 ceph-mon[35122]: pgmap v510: 129 pgs: 129 active+clean; 311 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-05T02:07:46.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:46 smithi139 ceph-mon[34262]: pgmap v510: 129 pgs: 129 active+clean; 311 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-05T02:07:46.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:46 smithi103 ceph-mon[30902]: pgmap v510: 129 pgs: 129 active+clean; 311 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-05T02:07:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.489+0000 7f37d39fd640 1 -- 172.21.15.103:0/1834803634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc101b60 msgr2=0x7f37cc101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.489+0000 7f37d39fd640 1 --2- 172.21.15.103:0/1834803634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc101b60 0x7f37cc101f60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009860 tx=0x7f37c802a340 comp rx=0 tx=0).stop 2024-09-05T02:07:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.489+0000 7f37d39fd640 1 -- 172.21.15.103:0/1834803634 shutdown_connections 2024-09-05T02:07:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.489+0000 7f37d39fd640 1 --2- 172.21.15.103:0/1834803634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37cc103710 0x7f37cc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.489+0000 7f37d39fd640 1 --2- 172.21.15.103:0/1834803634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37cc102d50 0x7f37cc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.489+0000 7f37d39fd640 1 --2- 172.21.15.103:0/1834803634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc101b60 0x7f37cc101f60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.489+0000 7f37d39fd640 1 -- 172.21.15.103:0/1834803634 >> 172.21.15.103:0/1834803634 conn(0x7f37cc0fd2e0 msgr2=0x7f37cc0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.489+0000 7f37d39fd640 1 -- 172.21.15.103:0/1834803634 shutdown_connections 2024-09-05T02:07:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.489+0000 7f37d39fd640 1 -- 172.21.15.103:0/1834803634 wait complete. 2024-09-05T02:07:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.490+0000 7f37d39fd640 1 Processor -- start 2024-09-05T02:07:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.490+0000 7f37d39fd640 1 -- start start 2024-09-05T02:07:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.490+0000 7f37d39fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37cc101b60 0x7f37cc197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.490+0000 7f37d39fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37cc102d50 0x7f37cc198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.490+0000 7f37d39fd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc103710 0x7f37cc19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.490+0000 7f37d39fd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc10d690 con 0x7f37cc101b60 2024-09-05T02:07:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.490+0000 7f37d39fd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc10d510 con 0x7f37cc102d50 2024-09-05T02:07:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.490+0000 7f37d39fd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc10d810 con 0x7f37cc103710 2024-09-05T02:07:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1f9a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc103710 0x7f37cc19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1f9a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc103710 0x7f37cc19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:45132/0 (socket says 172.21.15.103:45132) 2024-09-05T02:07:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1f9a640 1 -- 172.21.15.103:0/3268421106 learned_addr learned my addr 172.21.15.103:0/3268421106 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d0f98640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37cc102d50 0x7f37cc198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1799640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37cc101b60 0x7f37cc197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1f9a640 1 -- 172.21.15.103:0/3268421106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37cc102d50 msgr2=0x7f37cc198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1f9a640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37cc102d50 0x7f37cc198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1f9a640 1 -- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37cc101b60 msgr2=0x7f37cc197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1f9a640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37cc101b60 0x7f37cc197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1f9a640 1 -- 172.21.15.103:0/3268421106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc19f960 con 0x7f37cc103710 2024-09-05T02:07:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d0f98640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37cc102d50 0x7f37cc198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d1f9a640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc103710 0x7f37cc19f2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f37c400eb10 tx=0x7f37c400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37c27fc640 1 -- 172.21.15.103:0/3268421106 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c400ce20 con 0x7f37cc103710 2024-09-05T02:07:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37c27fc640 1 -- 172.21.15.103:0/3268421106 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37c4005020 con 0x7f37cc103710 2024-09-05T02:07:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.491+0000 7f37d39fd640 1 -- 172.21.15.103:0/3268421106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc19fc50 con 0x7f37cc103710 2024-09-05T02:07:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.492+0000 7f37c27fc640 1 -- 172.21.15.103:0/3268421106 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4010690 con 0x7f37cc103710 2024-09-05T02:07:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.492+0000 7f37d39fd640 1 -- 172.21.15.103:0/3268421106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc1a01a0 con 0x7f37cc103710 2024-09-05T02:07:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d687d740 1 Processor -- start 2024-09-05T02:07:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37c27fc640 1 -- 172.21.15.103:0/3268421106 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f37c40051d0 con 0x7f37cc103710 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d687d740 1 -- start start 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d687d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d322da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d687d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d327da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d687d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 0x55d34d32a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d687d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d34d2862f0 con 0x55d34d3232e0 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d687d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d34d22e270 con 0x55d34d3229a0 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d687d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d34d2822b0 con 0x55d34d328370 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d1799640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d327da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d1799640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d327da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41806/0 (socket says 172.21.15.103:41806) 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d1799640 1 -- 172.21.15.103:0/2430690319 learned_addr learned my addr 172.21.15.103:0/2430690319 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d0f98640 1 --2- 172.21.15.103:0/2430690319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 0x55d34d32a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.495+0000 7f37d0f98640 1 --2- 172.21.15.103:0/2430690319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d322da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37d1799640 1 -- 172.21.15.103:0/2430690319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 msgr2=0x55d34d32a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37d1799640 1 --2- 172.21.15.103:0/2430690319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 0x55d34d32a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37d1799640 1 -- 172.21.15.103:0/2430690319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 msgr2=0x55d34d322da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37d1799640 1 --2- 172.21.15.103:0/2430690319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d322da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37d1799640 1 -- 172.21.15.103:0/2430690319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34d22d340 con 0x55d34d3232e0 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37c27fc640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37b80a2c30 0x7f37b80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37d0f98640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37b80a2c30 0x7f37b80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37d1799640 1 --2- 172.21.15.103:0/2430690319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d327da0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009860 tx=0x7f37c8005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f12595c9f31311 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37c27fc640 1 -- 172.21.15.103:0/3268421106 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f37c40cfb00 con 0x7f37cc103710 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.496+0000 7f37b77fe640 1 -- 172.21.15.103:0/2430690319 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8037040 con 0x55d34d3232e0 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37b77fe640 1 -- 172.21.15.103:0/2430690319 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37c8038040 con 0x55d34d3232e0 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37b77fe640 1 -- 172.21.15.103:0/2430690319 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8003850 con 0x55d34d3232e0 2024-09-05T02:07:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d687d740 1 -- 172.21.15.103:0/2430690319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 msgr2=0x55d34d327da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d687d740 1 --2- 172.21.15.103:0/2430690319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d327da0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009860 tx=0x7f37c8005b20 comp rx=0 tx=0).stop 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d0f98640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37b80a2c30 0x7f37b80a50f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0058c0 tx=0x7f37bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d687d740 1 -- 172.21.15.103:0/2430690319 shutdown_connections 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d687d740 1 --2- 172.21.15.103:0/2430690319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 0x55d34d32a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d687d740 1 --2- 172.21.15.103:0/2430690319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d327da0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d687d740 1 --2- 172.21.15.103:0/2430690319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d322da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d687d740 1 -- 172.21.15.103:0/2430690319 >> 172.21.15.103:0/2430690319 conn(0x55d34d31e010 msgr2=0x55d34d31e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d687d740 1 -- 172.21.15.103:0/2430690319 shutdown_connections 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.497+0000 7f37d687d740 1 -- 172.21.15.103:0/2430690319 wait complete. 2024-09-05T02:07:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.498+0000 7f37d687d740 1 -- start start 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.498+0000 7f37d687d740 1 Processor -- start 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.498+0000 7f37d687d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d32cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.498+0000 7f37d687d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 0x55d34d32d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.498+0000 7f37d687d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 0x55d34d3340e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.498+0000 7f37d687d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d34d32b2e0 con 0x55d34d3229a0 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.498+0000 7f37d687d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d34d20f290 con 0x55d34d328370 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d687d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d34d32b460 con 0x55d34d3232e0 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1799640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d32cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1799640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d32cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41810/0 (socket says 172.21.15.103:41810) 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1799640 1 -- 172.21.15.103:0/1416979781 learned_addr learned my addr 172.21.15.103:0/1416979781 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d687d740 10 client.0 _ll_register_callbacks cb 0x55d34d240140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d0f98640 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 0x55d34d32d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1f9a640 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 0x55d34d3340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1799640 1 -- 172.21.15.103:0/1416979781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 msgr2=0x55d34d32d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1799640 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 0x55d34d32d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1799640 1 -- 172.21.15.103:0/1416979781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 msgr2=0x55d34d3340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1799640 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 0x55d34d3340e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1799640 1 -- 172.21.15.103:0/1416979781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34cf99210 con 0x55d34d3229a0 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1f9a640 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 0x55d34d3340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d1799640 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d32cb60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f37c8035040 tx=0x7f37c8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde89ea2ee6f1ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37b4ff9640 1 -- 172.21.15.103:0/1416979781 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c804e070 con 0x55d34d3229a0 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.499+0000 7f37d687d740 1 -- 172.21.15.103:0/1416979781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c8038040 con 0x55d34d3229a0 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37b4ff9640 1 -- 172.21.15.103:0/1416979781 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f37c8032440 con 0x55d34d3229a0 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37b4ff9640 1 -- 172.21.15.103:0/1416979781 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f37c8059050 con 0x55d34d3229a0 2024-09-05T02:07:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37d687d740 10 client.16038 fetch_fsmap learned FSMap version 35 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37d687d740 1 -- 172.21.15.103:0/1416979781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d34d065620 con 0x55d34d3229a0 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37b4ff9640 1 -- 172.21.15.103:0/1416979781 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37c8038040 con 0x55d34d3229a0 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37b4ff9640 1 -- 172.21.15.103:0/1416979781 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f37c803f730 con 0x55d34d3229a0 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37d687d740 10 client.16038 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37d687d740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37d687d740 20 client.16038 populate_metadata read hostname 'smithi103' 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37d687d740 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d34d3383e0 0x55d34d33a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37d687d740 4 client.16038 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37d687d740 1 -- 172.21.15.103:0/1416979781 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d34d293a20 con 0x55d34d3383e0 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.500+0000 7f37d0f98640 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d34d3383e0 0x55d34d33a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.501+0000 7f37b4ff9640 10 client.16038 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.501+0000 7f37d0f98640 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d34d3383e0 0x55d34d33a7d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b9396fb1f0042e server_cookie=ab035edba8867a8e in_seq=0 out_seq=0 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.501+0000 7f37b4ff9640 1 -- 172.21.15.103:0/1416979781 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d34d293a20 con 0x55d34d3383e0 2024-09-05T02:07:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.501+0000 7f37b4ff9640 10 client.16038 handle_command_reply: tid=0 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 client.16038 shutdown 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 -- 172.21.15.103:0/1416979781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 msgr2=0x55d34d32cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d32cb60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f37c8035040 tx=0x7f37c8004610 comp rx=0 tx=0).stop 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 -- 172.21.15.103:0/1416979781 shutdown_connections 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d34d3383e0 0x55d34d33a7d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 0x55d34d3340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 0x55d34d32d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37b4ff9640 0 client.16038 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 --2- 172.21.15.103:0/1416979781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d32cb60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 -- 172.21.15.103:0/1416979781 >> 172.21.15.103:0/1416979781 conn(0x55d34d31e010 msgr2=0x55d34d31f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 -- 172.21.15.103:0/1416979781 shutdown_connections 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 1 -- 172.21.15.103:0/1416979781 wait complete. 2024-09-05T02:07:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.502+0000 7f37d687d740 20 client.16038 trim_cache size 0 max 16384 2024-09-05T02:07:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d687d740 1 Processor -- start 2024-09-05T02:07:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d687d740 1 -- start start 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d687d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d31d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d687d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d3340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d687d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 0x55d34d31f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d687d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1aa710 con 0x55d34d3232e0 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d687d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1ac310 con 0x55d34d3229a0 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d687d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1aade0 con 0x55d34d328370 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d1799640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d31d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d1799640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d31d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:48006/0 (socket says 172.21.15.103:48006) 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d1799640 1 -- 172.21.15.103:0/2503761338 learned_addr learned my addr 172.21.15.103:0/2503761338 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d1f9a640 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 0x55d34d31f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d0f98640 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d3340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d1799640 1 -- 172.21.15.103:0/2503761338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 msgr2=0x55d34d31f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d1799640 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 0x55d34d31f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d1799640 1 -- 172.21.15.103:0/2503761338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 msgr2=0x55d34d3340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d1799640 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d3340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.526+0000 7f37d1799640 1 -- 172.21.15.103:0/2503761338 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34d33ae30 con 0x55d34d3229a0 2024-09-05T02:07:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.527+0000 7f37d0f98640 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d3340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:07:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.527+0000 7f37d1799640 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d31d0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37c802a860 tx=0x7f37c8005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9262ec38ef7bd52d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.527+0000 7f37b6ffd640 1 -- 172.21.15.103:0/2503761338 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c804b070 con 0x55d34d3229a0 2024-09-05T02:07:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.527+0000 7f37b6ffd640 1 -- 172.21.15.103:0/2503761338 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37c8009450 con 0x55d34d3229a0 2024-09-05T02:07:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.527+0000 7f37b6ffd640 1 -- 172.21.15.103:0/2503761338 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c804e040 con 0x55d34d3229a0 2024-09-05T02:07:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.528+0000 7f37d687d740 1 -- 172.21.15.103:0/2503761338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 msgr2=0x55d34d31d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.528+0000 7f37d687d740 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d31d0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37c802a860 tx=0x7f37c8005f10 comp rx=0 tx=0).stop 2024-09-05T02:07:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.528+0000 7f37d687d740 1 -- 172.21.15.103:0/2503761338 shutdown_connections 2024-09-05T02:07:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.528+0000 7f37d687d740 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d328370 0x55d34d31f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.528+0000 7f37d687d740 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3232e0 0x55d34d3340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.528+0000 7f37d687d740 1 --2- 172.21.15.103:0/2503761338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d3229a0 0x55d34d31d0b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.528+0000 7f37d687d740 1 -- 172.21.15.103:0/2503761338 >> 172.21.15.103:0/2503761338 conn(0x55d34d31e010 msgr2=0x55d34d31c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.528+0000 7f37d687d740 1 -- 172.21.15.103:0/2503761338 shutdown_connections 2024-09-05T02:07:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.528+0000 7f37d687d740 1 -- 172.21.15.103:0/2503761338 wait complete. 2024-09-05T02:07:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d687d740 1 -- start start 2024-09-05T02:07:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d687d740 1 Processor -- start 2024-09-05T02:07:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d687d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d331280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d687d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 0x55d34d3317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d687d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 0x55d34d3327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d1799640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d331280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d687d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1abc40 con 0x55d34d3229a0 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d687d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1ab4b0 con 0x55d34d328370 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d1799640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d331280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41834/0 (socket says 172.21.15.103:41834) 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d1799640 1 -- 172.21.15.103:0/4124205796 learned_addr learned my addr 172.21.15.103:0/4124205796 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d0f98640 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 0x55d34d3317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d687d740 1 -- 172.21.15.103:0/4124205796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1a86f0 con 0x55d34d3232e0 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.529+0000 7f37d687d740 10 client.0 _ll_register_callbacks cb 0x7f37bc011650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37d1f9a640 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 0x55d34d3327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37d0f98640 1 -- 172.21.15.103:0/4124205796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 msgr2=0x55d34d3327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37d0f98640 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 0x55d34d3327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37d0f98640 1 -- 172.21.15.103:0/4124205796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 msgr2=0x55d34d331280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37d0f98640 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d331280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37d0f98640 1 -- 172.21.15.103:0/4124205796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34d332de0 con 0x55d34d3232e0 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37d1799640 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d331280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37d0f98640 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 0x55d34d3317c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55d34d33b060 tx=0x7f37bc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232440a3b58f011b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37b4ff9640 1 -- 172.21.15.103:0/4124205796 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc020590 con 0x55d34d3232e0 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37b4ff9640 1 -- 172.21.15.103:0/4124205796 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f37bc020730 con 0x55d34d3232e0 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.530+0000 7f37d687d740 1 -- 172.21.15.103:0/4124205796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c8009450 con 0x55d34d3232e0 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37b4ff9640 1 -- 172.21.15.103:0/4124205796 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f37bc020930 con 0x55d34d3232e0 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37d687d740 10 client.25408 fetch_fsmap learned FSMap version 35 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37d687d740 1 -- 172.21.15.103:0/4124205796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d34d3341c0 con 0x55d34d3232e0 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37b4ff9640 1 -- 172.21.15.103:0/4124205796 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37c8009450 con 0x55d34d3232e0 2024-09-05T02:07:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37b4ff9640 1 -- 172.21.15.103:0/4124205796 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f37bc037a20 con 0x55d34d3232e0 2024-09-05T02:07:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37d687d740 10 client.25408 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:07:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37d687d740 10 client.25408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:07:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37d687d740 20 client.25408 populate_metadata read hostname 'smithi103' 2024-09-05T02:07:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37d687d740 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d34d3a35b0 0x55d34d3a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:07:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37d687d740 4 client.25408 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:07:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37d687d740 1 -- 172.21.15.103:0/4124205796 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d34d28dd70 con 0x55d34d3a35b0 2024-09-05T02:07:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.531+0000 7f37d1799640 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d34d3a35b0 0x55d34d3a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:07:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.532+0000 7f37b4ff9640 10 client.25408 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.532+0000 7f37d1799640 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d34d3a35b0 0x55d34d3a59a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1ec8ec4db728595 server_cookie=9c9602578691dff4 in_seq=0 out_seq=0 2024-09-05T02:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37b4ff9640 1 -- 172.21.15.103:0/4124205796 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d34d28dd70 con 0x55d34d3a35b0 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37b4ff9640 10 client.25408 handle_command_reply: tid=0 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 client.25408 shutdown 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 -- 172.21.15.103:0/4124205796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 msgr2=0x55d34d3317c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 0x55d34d3317c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55d34d33b060 tx=0x7f37bc004730 comp rx=0 tx=0).stop 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 -- 172.21.15.103:0/4124205796 shutdown_connections 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55d34d3a35b0 0x55d34d3a59a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d34d328370 0x55d34d3327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34d3232e0 0x55d34d3317c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 --2- 172.21.15.103:0/4124205796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d34d3229a0 0x55d34d331280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37b4ff9640 0 client.25408 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:07:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 -- 172.21.15.103:0/4124205796 >> 172.21.15.103:0/4124205796 conn(0x55d34d31e010 msgr2=0x55d34d31d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 -- 172.21.15.103:0/4124205796 shutdown_connections 2024-09-05T02:07:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.625+0000 7f37d687d740 1 -- 172.21.15.103:0/4124205796 wait complete. 2024-09-05T02:07:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.626+0000 7f37d687d740 20 client.25408 trim_cache size 0 max 16384 2024-09-05T02:07:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:07:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:07:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:07:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:07:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.626+0000 7f37d39fd640 1 -- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37b80a2c30 msgr2=0x7f37b80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.626+0000 7f37d39fd640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37b80a2c30 0x7f37b80a50f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0058c0 tx=0x7f37bc009260 comp rx=0 tx=0).stop 2024-09-05T02:07:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.626+0000 7f37d39fd640 1 -- 172.21.15.103:0/3268421106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc103710 msgr2=0x7f37cc19f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:07:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.626+0000 7f37d39fd640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc103710 0x7f37cc19f2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f37c400eb10 tx=0x7f37c400ef10 comp rx=0 tx=0).stop 2024-09-05T02:07:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.627+0000 7f37d39fd640 1 -- 172.21.15.103:0/3268421106 shutdown_connections 2024-09-05T02:07:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.627+0000 7f37d39fd640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f37b80a2c30 0x7f37b80a50f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.627+0000 7f37d39fd640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37cc103710 0x7f37cc19f2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.627+0000 7f37d39fd640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37cc102d50 0x7f37cc198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.627+0000 7f37d39fd640 1 --2- 172.21.15.103:0/3268421106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37cc101b60 0x7f37cc197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:07:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.627+0000 7f37d39fd640 1 -- 172.21.15.103:0/3268421106 >> 172.21.15.103:0/3268421106 conn(0x7f37cc0fd2e0 msgr2=0x7f37cc106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:07:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.627+0000 7f37d39fd640 1 -- 172.21.15.103:0/3268421106 shutdown_connections 2024-09-05T02:07:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:07:48.627+0000 7f37d39fd640 1 -- 172.21.15.103:0/3268421106 wait complete. 2024-09-05T02:07:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:48 smithi167 ceph-mon[35122]: pgmap v511: 129 pgs: 129 active+clean; 312 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-05T02:07:48.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:48 smithi139 ceph-mon[34262]: pgmap v511: 129 pgs: 129 active+clean; 312 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-05T02:07:48.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:48 smithi103 ceph-mon[30902]: pgmap v511: 129 pgs: 129 active+clean; 312 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-05T02:07:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:50 smithi167 ceph-mon[35122]: pgmap v512: 129 pgs: 129 active+clean; 315 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-09-05T02:07:50.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:50 smithi139 ceph-mon[34262]: pgmap v512: 129 pgs: 129 active+clean; 315 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-09-05T02:07:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:50 smithi103 ceph-mon[30902]: pgmap v512: 129 pgs: 129 active+clean; 315 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-09-05T02:07:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:52 smithi167 ceph-mon[35122]: pgmap v513: 129 pgs: 129 active+clean; 319 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-05T02:07:52.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:52 smithi139 ceph-mon[34262]: pgmap v513: 129 pgs: 129 active+clean; 319 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-05T02:07:52.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:52 smithi103 ceph-mon[30902]: pgmap v513: 129 pgs: 129 active+clean; 319 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-05T02:07:53.875 INFO:tasks.workunit.client.0.smithi103.stdout: 59 12681 107 7820 113 4130 70 2024-09-05T02:07:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:54 smithi167 ceph-mon[35122]: pgmap v514: 129 pgs: 129 active+clean; 324 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-09-05T02:07:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:54 smithi139 ceph-mon[34262]: pgmap v514: 129 pgs: 129 active+clean; 324 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-09-05T02:07:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:54 smithi103 ceph-mon[30902]: pgmap v514: 129 pgs: 129 active+clean; 324 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-09-05T02:07:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:56 smithi167 ceph-mon[35122]: pgmap v515: 129 pgs: 129 active+clean; 327 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-09-05T02:07:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:56 smithi139 ceph-mon[34262]: pgmap v515: 129 pgs: 129 active+clean; 327 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-09-05T02:07:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:56 smithi103 ceph-mon[30902]: pgmap v515: 129 pgs: 129 active+clean; 327 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-09-05T02:07:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:07:58 smithi167 ceph-mon[35122]: pgmap v516: 129 pgs: 129 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-05T02:07:58.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:07:58 smithi139 ceph-mon[34262]: pgmap v516: 129 pgs: 129 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-05T02:07:58.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:07:58 smithi103 ceph-mon[30902]: pgmap v516: 129 pgs: 129 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-05T02:08:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:00 smithi167 ceph-mon[35122]: pgmap v517: 129 pgs: 129 active+clean; 333 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-05T02:08:00.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:00 smithi139 ceph-mon[34262]: pgmap v517: 129 pgs: 129 active+clean; 333 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-05T02:08:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:00 smithi103 ceph-mon[30902]: pgmap v517: 129 pgs: 129 active+clean; 333 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-05T02:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.750+0000 7f3642c8d640 1 -- 172.21.15.103:0/2671012873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c100a50 msgr2=0x7f363c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.750+0000 7f3642c8d640 1 --2- 172.21.15.103:0/2671012873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c100a50 0x7f363c100e50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f363002a3f0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.750+0000 7f3642c8d640 1 -- 172.21.15.103:0/2671012873 shutdown_connections 2024-09-05T02:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.750+0000 7f3642c8d640 1 --2- 172.21.15.103:0/2671012873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f363c102600 0x7f363c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.750+0000 7f3642c8d640 1 --2- 172.21.15.103:0/2671012873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c101c40 0x7f363c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.750+0000 7f3642c8d640 1 --2- 172.21.15.103:0/2671012873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c100a50 0x7f363c100e50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.750+0000 7f3642c8d640 1 -- 172.21.15.103:0/2671012873 >> 172.21.15.103:0/2671012873 conn(0x7f363c0fc1b0 msgr2=0x7f363c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.750+0000 7f3642c8d640 1 -- 172.21.15.103:0/2671012873 shutdown_connections 2024-09-05T02:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.750+0000 7f3642c8d640 1 -- 172.21.15.103:0/2671012873 wait complete. 2024-09-05T02:08:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f3642c8d640 1 Processor -- start 2024-09-05T02:08:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f3642c8d640 1 -- start start 2024-09-05T02:08:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f3642c8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c100a50 0x7f363c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f3642c8d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f363c101c40 0x7f363c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f3642c8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c102600 0x7f363c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f3642c8d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10c580 con 0x7f363c102600 2024-09-05T02:08:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f3642c8d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10c400 con 0x7f363c100a50 2024-09-05T02:08:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f3642c8d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10c700 con 0x7f363c101c40 2024-09-05T02:08:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f364122a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c102600 0x7f363c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f364122a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c102600 0x7f363c19f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41324/0 (socket says 172.21.15.103:41324) 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f364122a640 1 -- 172.21.15.103:0/4255904301 learned_addr learned my addr 172.21.15.103:0/4255904301 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f363bfff640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f363c101c40 0x7f363c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f3640a29640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c100a50 0x7f363c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f363bfff640 1 -- 172.21.15.103:0/4255904301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c100a50 msgr2=0x7f363c197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f363bfff640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c100a50 0x7f363c197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f363bfff640 1 -- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c102600 msgr2=0x7f363c19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f363bfff640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c102600 0x7f363c19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f363bfff640 1 -- 172.21.15.103:0/4255904301 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c19f9b0 con 0x7f363c101c40 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.751+0000 7f364122a640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c102600 0x7f363c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.752+0000 7f3640a29640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c100a50 0x7f363c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.752+0000 7f363bfff640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f363c101c40 0x7f363c1982e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f362800e9f0 tx=0x7f362800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.752+0000 7f3639ffb640 1 -- 172.21.15.103:0/4255904301 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362800cd80 con 0x7f363c101c40 2024-09-05T02:08:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.752+0000 7f3642c8d640 1 -- 172.21.15.103:0/4255904301 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c19fca0 con 0x7f363c101c40 2024-09-05T02:08:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.752+0000 7f3639ffb640 1 -- 172.21.15.103:0/4255904301 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36280035f0 con 0x7f363c101c40 2024-09-05T02:08:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.752+0000 7f3639ffb640 1 -- 172.21.15.103:0/4255904301 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628010670 con 0x7f363c101c40 2024-09-05T02:08:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.752+0000 7f3642c8d640 1 -- 172.21.15.103:0/4255904301 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1a01a0 con 0x7f363c101c40 2024-09-05T02:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f3639ffb640 1 -- 172.21.15.103:0/4255904301 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f36280037a0 con 0x7f363c101c40 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f3645a29740 1 Processor -- start 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f3645a29740 1 -- start start 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f3645a29740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f3645a29740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b37cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f3645a29740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 0x55f74b37f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f3645a29740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f74b2db2f0 con 0x55f74b37d370 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f3645a29740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f74b283270 con 0x55f74b3779a0 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f3645a29740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f74b2d72b0 con 0x55f74b3782e0 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f363bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 0x55f74b37f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.755+0000 7f364122a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f3640a29640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b37cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f364122a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:41908/0 (socket says 172.21.15.103:41908) 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f364122a640 1 -- 172.21.15.103:0/1387577501 learned_addr learned my addr 172.21.15.103:0/1387577501 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f364122a640 1 -- 172.21.15.103:0/1387577501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 msgr2=0x55f74b37cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f364122a640 1 --2- 172.21.15.103:0/1387577501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b37cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f364122a640 1 -- 172.21.15.103:0/1387577501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 msgr2=0x55f74b37f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f364122a640 1 --2- 172.21.15.103:0/1387577501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 0x55f74b37f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f364122a640 1 -- 172.21.15.103:0/1387577501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74b282340 con 0x55f74b3779a0 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f3639ffb640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f362c0a2cb0 0x7f362c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.756+0000 7f364122a640 1 --2- 172.21.15.103:0/1387577501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36340118b0 tx=0x7f3634011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6e7d1e85f70240c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.757+0000 7f3640a29640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f362c0a2cb0 0x7f362c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.757+0000 7f3639ffb640 1 -- 172.21.15.103:0/4255904301 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f36280cf730 con 0x7f363c101c40 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.757+0000 7f3626ffd640 1 -- 172.21.15.103:0/1387577501 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363401e070 con 0x55f74b3779a0 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.757+0000 7f3626ffd640 1 -- 172.21.15.103:0/1387577501 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f363400fe70 con 0x55f74b3779a0 2024-09-05T02:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.757+0000 7f3626ffd640 1 -- 172.21.15.103:0/1387577501 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634019d00 con 0x55f74b3779a0 2024-09-05T02:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 -- 172.21.15.103:0/1387577501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 msgr2=0x55f74b377da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 --2- 172.21.15.103:0/1387577501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36340118b0 tx=0x7f3634011d70 comp rx=0 tx=0).stop 2024-09-05T02:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3640a29640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f362c0a2cb0 0x7f362c0a5170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3630009230 tx=0x7f36300314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 -- 172.21.15.103:0/1387577501 shutdown_connections 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 --2- 172.21.15.103:0/1387577501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 0x55f74b37f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 --2- 172.21.15.103:0/1387577501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b37cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 --2- 172.21.15.103:0/1387577501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377da0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 -- 172.21.15.103:0/1387577501 >> 172.21.15.103:0/1387577501 conn(0x55f74b373010 msgr2=0x55f74b373410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 -- 172.21.15.103:0/1387577501 shutdown_connections 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 -- 172.21.15.103:0/1387577501 wait complete. 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 -- start start 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.758+0000 7f3645a29740 1 Processor -- start 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3645a29740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b381b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3645a29740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b382050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3645a29740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 0x55f74b389090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3645a29740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f74b380340 con 0x55f74b37d370 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3645a29740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f74b264290 con 0x55f74b3779a0 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3645a29740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f74b3804c0 con 0x55f74b3782e0 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3645a29740 10 client.0 _ll_register_callbacks cb 0x55f74b295140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f363bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b382050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3640a29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b381b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3640a29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b381b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:41924/0 (socket says 172.21.15.103:41924) 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f363bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b382050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:33346/0 (socket says 172.21.15.103:33346) 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f3640a29640 1 -- 172.21.15.103:0/2057458386 learned_addr learned my addr 172.21.15.103:0/2057458386 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.759+0000 7f364122a640 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 0x55f74b389090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f3640a29640 1 -- 172.21.15.103:0/2057458386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 msgr2=0x55f74b382050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f3640a29640 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b382050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f3640a29640 1 -- 172.21.15.103:0/2057458386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 msgr2=0x55f74b389090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f3640a29640 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 0x55f74b389090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f3640a29640 1 -- 172.21.15.103:0/2057458386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74afee210 con 0x55f74b3779a0 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f364122a640 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 0x55f74b389090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f3640a29640 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b381b10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f363c101aa0 tx=0x7f3630009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d516a6ee3005348 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f361bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f361bfff640 1 -- 172.21.15.103:0/2057458386 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363004d070 con 0x55f74b3779a0 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f361bfff640 1 -- 172.21.15.103:0/2057458386 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3630004460 con 0x55f74b3779a0 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f3645a29740 1 -- 172.21.15.103:0/2057458386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f363400fe70 con 0x55f74b3779a0 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f361bfff640 1 -- 172.21.15.103:0/2057458386 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3630059050 con 0x55f74b3779a0 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f3645a29740 10 client.25481 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f3645a29740 1 -- 172.21.15.103:0/2057458386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f74b0ba620 con 0x55f74b3779a0 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.760+0000 7f361bfff640 1 -- 172.21.15.103:0/2057458386 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f363400fe70 con 0x55f74b3779a0 2024-09-05T02:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f361bfff640 1 -- 172.21.15.103:0/2057458386 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f3630003b20 con 0x55f74b3779a0 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f3645a29740 10 client.25481 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f3645a29740 10 client.25481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f3645a29740 20 client.25481 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f3645a29740 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f74b391b50 0x55f74b393fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f3645a29740 4 client.25481 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f3645a29740 1 -- 172.21.15.103:0/2057458386 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f74b2e8a20 con 0x55f74b391b50 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f363bfff640 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f74b391b50 0x55f74b393fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f363bfff640 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f74b391b50 0x55f74b393fc0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0737ecc1b47cd27 server_cookie=8a1b56cde954dac1 in_seq=0 out_seq=0 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.761+0000 7f361bfff640 10 client.25481 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f361bfff640 1 -- 172.21.15.103:0/2057458386 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f74b2e8a20 con 0x55f74b391b50 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f361bfff640 10 client.25481 handle_command_reply: tid=0 2024-09-05T02:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 client.25481 shutdown 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 -- 172.21.15.103:0/2057458386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 msgr2=0x55f74b381b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b381b10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f363c101aa0 tx=0x7f3630009260 comp rx=0 tx=0).stop 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 -- 172.21.15.103:0/2057458386 shutdown_connections 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f74b391b50 0x55f74b393fc0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b37d370 0x55f74b389090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b382050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 --2- 172.21.15.103:0/2057458386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b381b10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f361bfff640 0 client.25481 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 -- 172.21.15.103:0/2057458386 >> 172.21.15.103:0/2057458386 conn(0x55f74b373010 msgr2=0x55f74b374820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 -- 172.21.15.103:0/2057458386 shutdown_connections 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.762+0000 7f3645a29740 1 -- 172.21.15.103:0/2057458386 wait complete. 2024-09-05T02:08:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.763+0000 7f3645a29740 20 client.25481 trim_cache size 0 max 16384 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3645a29740 1 Processor -- start 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3645a29740 1 -- start start 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3645a29740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3645a29740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3782e0 0x55f74b388fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3645a29740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b37d370 0x55f74b3720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3645a29740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1a8480 con 0x55f74b3782e0 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3645a29740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1aa080 con 0x55f74b3779a0 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3645a29740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1a8b50 con 0x55f74b37d370 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3640a29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f363bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3782e0 0x55f74b388fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3640a29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:41934/0 (socket says 172.21.15.103:41934) 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f3640a29640 1 -- 172.21.15.103:0/2935966170 learned_addr learned my addr 172.21.15.103:0/2935966170 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.789+0000 7f364122a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b37d370 0x55f74b3720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.790+0000 7f3640a29640 1 -- 172.21.15.103:0/2935966170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b37d370 msgr2=0x55f74b3720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.790+0000 7f3640a29640 1 --2- 172.21.15.103:0/2935966170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b37d370 0x55f74b3720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.790+0000 7f3640a29640 1 -- 172.21.15.103:0/2935966170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3782e0 msgr2=0x55f74b388fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.790+0000 7f3640a29640 1 --2- 172.21.15.103:0/2935966170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3782e0 0x55f74b388fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.790+0000 7f3640a29640 1 -- 172.21.15.103:0/2935966170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74b37c700 con 0x55f74b3779a0 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.790+0000 7f3640a29640 1 --2- 172.21.15.103:0/2935966170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f363002a910 tx=0x7f363003d830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=484b9f65986e5498 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.790+0000 7f36267fc640 1 -- 172.21.15.103:0/2935966170 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363004d070 con 0x55f74b3779a0 2024-09-05T02:08:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.790+0000 7f36267fc640 1 -- 172.21.15.103:0/2935966170 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3630062070 con 0x55f74b3779a0 2024-09-05T02:08:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.790+0000 7f36267fc640 1 -- 172.21.15.103:0/2935966170 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630035040 con 0x55f74b3779a0 2024-09-05T02:08:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 -- 172.21.15.103:0/2935966170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 msgr2=0x55f74b377d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 --2- 172.21.15.103:0/2935966170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f363002a910 tx=0x7f363003d830 comp rx=0 tx=0).stop 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 -- 172.21.15.103:0/2935966170 shutdown_connections 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 --2- 172.21.15.103:0/2935966170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b37d370 0x55f74b3720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 --2- 172.21.15.103:0/2935966170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3782e0 0x55f74b388fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 --2- 172.21.15.103:0/2935966170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b3779a0 0x55f74b377d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 -- 172.21.15.103:0/2935966170 >> 172.21.15.103:0/2935966170 conn(0x55f74b373010 msgr2=0x55f74b3719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 -- 172.21.15.103:0/2935966170 shutdown_connections 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 -- 172.21.15.103:0/2935966170 wait complete. 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 -- start start 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.791+0000 7f3645a29740 1 Processor -- start 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3645a29740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3779a0 0x55f74b386400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3645a29740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b386940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3645a29740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b37d370 0x55f74b387950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3645a29740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1a99b0 con 0x55f74b3779a0 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3645a29740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1a9220 con 0x55f74b37d370 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3645a29740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1a6460 con 0x55f74b3782e0 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3645a29740 10 client.0 _ll_register_callbacks cb 0x7f36300317b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3640a29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3779a0 0x55f74b386400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f363bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b386940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3640a29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3779a0 0x55f74b386400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41376/0 (socket says 172.21.15.103:41376) 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f3640a29640 1 -- 172.21.15.103:0/3371016178 learned_addr learned my addr 172.21.15.103:0/3371016178 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f364122a640 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b37d370 0x55f74b387950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f363bfff640 1 -- 172.21.15.103:0/3371016178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b37d370 msgr2=0x55f74b387950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f363bfff640 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b37d370 0x55f74b387950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f363bfff640 1 -- 172.21.15.103:0/3371016178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3779a0 msgr2=0x55f74b386400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f363bfff640 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3779a0 0x55f74b386400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.792+0000 7f363bfff640 1 -- 172.21.15.103:0/3371016178 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74b387f60 con 0x55f74b3782e0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f363bfff640 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b386940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f362809b880 tx=0x7f36280cce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9cf36077e8ee493 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f361bfff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f361bfff640 1 -- 172.21.15.103:0/3371016178 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362809c280 con 0x55f74b3782e0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f3645a29740 1 -- 172.21.15.103:0/3371016178 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3630062070 con 0x55f74b3782e0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f361bfff640 1 -- 172.21.15.103:0/3371016178 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3628009310 con 0x55f74b3782e0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f361bfff640 1 -- 172.21.15.103:0/3371016178 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f362809bb80 con 0x55f74b3782e0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f3645a29740 10 client.25426 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f3645a29740 1 -- 172.21.15.103:0/3371016178 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f74b389340 con 0x55f74b3782e0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f361bfff640 1 -- 172.21.15.103:0/3371016178 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36280b3080 con 0x55f74b3782e0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.793+0000 7f361bfff640 1 -- 172.21.15.103:0/3371016178 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f3628098160 con 0x55f74b3782e0 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.794+0000 7f3645a29740 10 client.25426 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.794+0000 7f3645a29740 10 client.25426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.794+0000 7f3645a29740 20 client.25426 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.794+0000 7f3645a29740 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f74b3fcec0 0x55f74b3ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.794+0000 7f3645a29740 4 client.25426 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:08:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.794+0000 7f3645a29740 1 -- 172.21.15.103:0/3371016178 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f74b2e2d70 con 0x55f74b3fcec0 2024-09-05T02:08:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.794+0000 7f3640a29640 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f74b3fcec0 0x55f74b3ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.794+0000 7f361bfff640 10 client.25426 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.794+0000 7f3640a29640 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f74b3fcec0 0x55f74b3ff2e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9d81a334df98bf server_cookie=f79e8840b39075ec in_seq=0 out_seq=0 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f361bfff640 1 -- 172.21.15.103:0/3371016178 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f74b2e2d70 con 0x55f74b3fcec0 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f361bfff640 10 client.25426 handle_command_reply: tid=0 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 client.25426 shutdown 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 -- 172.21.15.103:0/3371016178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 msgr2=0x55f74b386940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b386940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f362809b880 tx=0x7f36280cce20 comp rx=0 tx=0).stop 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 -- 172.21.15.103:0/3371016178 shutdown_connections 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55f74b3fcec0 0x55f74b3ff2e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f74b37d370 0x55f74b387950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f74b3782e0 0x55f74b386940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 --2- 172.21.15.103:0/3371016178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f74b3779a0 0x55f74b386400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 -- 172.21.15.103:0/3371016178 >> 172.21.15.103:0/3371016178 conn(0x55f74b373010 msgr2=0x55f74b3721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f361bfff640 0 client.25426 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 -- 172.21.15.103:0/3371016178 shutdown_connections 2024-09-05T02:08:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 1 -- 172.21.15.103:0/3371016178 wait complete. 2024-09-05T02:08:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.847+0000 7f3645a29740 20 client.25426 trim_cache size 0 max 16384 2024-09-05T02:08:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:08:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:08:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:08:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:08:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 -- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f362c0a2cb0 msgr2=0x7f362c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f362c0a2cb0 0x7f362c0a5170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3630009230 tx=0x7f36300314a0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 -- 172.21.15.103:0/4255904301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f363c101c40 msgr2=0x7f363c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f363c101c40 0x7f363c1982e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f362800e9f0 tx=0x7f362800eeb0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 -- 172.21.15.103:0/4255904301 shutdown_connections 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f363c102600 0x7f363c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f362c0a2cb0 0x7f362c0a5170 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f363c101c40 0x7f363c1982e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 --2- 172.21.15.103:0/4255904301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c100a50 0x7f363c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 -- 172.21.15.103:0/4255904301 >> 172.21.15.103:0/4255904301 conn(0x7f363c0fc1b0 msgr2=0x7f363c06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 -- 172.21.15.103:0/4255904301 shutdown_connections 2024-09-05T02:08:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:01.848+0000 7f3642c8d640 1 -- 172.21.15.103:0/4255904301 wait complete. 2024-09-05T02:08:02.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:02 smithi167 ceph-mon[35122]: pgmap v518: 129 pgs: 129 active+clean; 334 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-05T02:08:02.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:02 smithi139 ceph-mon[34262]: pgmap v518: 129 pgs: 129 active+clean; 334 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-05T02:08:02.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:02 smithi103 ceph-mon[30902]: pgmap v518: 129 pgs: 129 active+clean; 334 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-05T02:08:03.875 INFO:tasks.workunit.client.0.smithi103.stdout: 59 12440 50 7652 57 4849 78 2024-09-05T02:08:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:04 smithi167 ceph-mon[35122]: pgmap v519: 129 pgs: 129 active+clean; 338 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-09-05T02:08:04.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:04 smithi139 ceph-mon[34262]: pgmap v519: 129 pgs: 129 active+clean; 338 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-09-05T02:08:04.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:04 smithi103 ceph-mon[30902]: pgmap v519: 129 pgs: 129 active+clean; 338 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-09-05T02:08:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:06 smithi167 ceph-mon[35122]: pgmap v520: 129 pgs: 129 active+clean; 340 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-09-05T02:08:06.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:06 smithi139 ceph-mon[34262]: pgmap v520: 129 pgs: 129 active+clean; 340 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-09-05T02:08:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:06 smithi103 ceph-mon[30902]: pgmap v520: 129 pgs: 129 active+clean; 340 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-09-05T02:08:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:08 smithi167 ceph-mon[35122]: pgmap v521: 129 pgs: 129 active+clean; 342 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-09-05T02:08:08.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:08 smithi139 ceph-mon[34262]: pgmap v521: 129 pgs: 129 active+clean; 342 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-09-05T02:08:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:08 smithi103 ceph-mon[30902]: pgmap v521: 129 pgs: 129 active+clean; 342 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-09-05T02:08:09.398 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.3 2024-09-05T02:08:09.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:08:09.756 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.302+0000 7fa642fed640 1 -- 172.21.15.103:0/965218776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa63c103fe0 msgr2=0x7fa63c1043c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.302+0000 7fa642fed640 1 --2- 172.21.15.103:0/965218776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa63c103fe0 0x7fa63c1043c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa630009900 tx=0x7fa63002a3f0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.302+0000 7fa642fed640 1 -- 172.21.15.103:0/965218776 shutdown_connections 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.302+0000 7fa642fed640 1 --2- 172.21.15.103:0/965218776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c109110 0x7fa63c110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.302+0000 7fa642fed640 1 --2- 172.21.15.103:0/965218776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63c104990 0x7fa63c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.302+0000 7fa642fed640 1 --2- 172.21.15.103:0/965218776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa63c103fe0 0x7fa63c1043c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.302+0000 7fa642fed640 1 -- 172.21.15.103:0/965218776 >> 172.21.15.103:0/965218776 conn(0x7fa63c0ffa60 msgr2=0x7fa63c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.302+0000 7fa642fed640 1 -- 172.21.15.103:0/965218776 shutdown_connections 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.302+0000 7fa642fed640 1 -- 172.21.15.103:0/965218776 wait complete. 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.303+0000 7fa642fed640 1 Processor -- start 2024-09-05T02:08:10.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.303+0000 7fa642fed640 1 -- start start 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.303+0000 7fa642fed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c103fe0 0x7fa63c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.303+0000 7fa642fed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa63c104990 0x7fa63c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.303+0000 7fa642fed640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63c109110 0x7fa63c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa642fed640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c113fc0 con 0x7fa63c104990 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa642fed640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c113e40 con 0x7fa63c109110 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa642fed640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c114140 con 0x7fa63c103fe0 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa640d89640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c103fe0 0x7fa63c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa640d89640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c103fe0 0x7fa63c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:41650/0 (socket says 172.21.15.103:41650) 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa640d89640 1 -- 172.21.15.103:0/870388903 learned_addr learned my addr 172.21.15.103:0/870388903 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa63bfff640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa63c104990 0x7fa63c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:10.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa64158a640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63c109110 0x7fa63c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa640d89640 1 -- 172.21.15.103:0/870388903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63c109110 msgr2=0x7fa63c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa640d89640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63c109110 0x7fa63c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa640d89640 1 -- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa63c104990 msgr2=0x7fa63c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa640d89640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa63c104990 0x7fa63c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa640d89640 1 -- 172.21.15.103:0/870388903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c19f4b0 con 0x7fa63c103fe0 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa63bfff640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa63c104990 0x7fa63c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.304+0000 7fa640d89640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c103fe0 0x7fa63c19a500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa630005940 tx=0x7fa6300040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.305+0000 7fa639ffb640 1 -- 172.21.15.103:0/870388903 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63003f030 con 0x7fa63c103fe0 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.305+0000 7fa639ffb640 1 -- 172.21.15.103:0/870388903 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6300044c0 con 0x7fa63c103fe0 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.305+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63c19f740 con 0x7fa63c103fe0 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.305+0000 7fa639ffb640 1 -- 172.21.15.103:0/870388903 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630035070 con 0x7fa63c103fe0 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.305+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63c1a7080 con 0x7fa63c103fe0 2024-09-05T02:08:10.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.306+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63c1043c0 con 0x7fa63c103fe0 2024-09-05T02:08:10.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.308+0000 7fa639ffb640 1 -- 172.21.15.103:0/870388903 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa630042030 con 0x7fa63c103fe0 2024-09-05T02:08:10.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.309+0000 7fa639ffb640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa62c0a2c00 0x7fa62c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:10.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.310+0000 7fa63bfff640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa62c0a2c00 0x7fa62c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:10.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.310+0000 7fa639ffb640 1 -- 172.21.15.103:0/870388903 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa6300efb20 con 0x7fa63c103fe0 2024-09-05T02:08:10.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.310+0000 7fa63bfff640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa62c0a2c00 0x7fa62c0a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6280095f0 tx=0x7fa628005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:10.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.312+0000 7fa639ffb640 1 -- 172.21.15.103:0/870388903 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa630049030 con 0x7fa63c103fe0 2024-09-05T02:08:10.436 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:10 smithi103 ceph-mon[30902]: pgmap v522: 129 pgs: 129 active+clean; 344 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-09-05T02:08:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:10 smithi167 ceph-mon[35122]: pgmap v522: 129 pgs: 129 active+clean; 344 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-09-05T02:08:10.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.643+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa63c10c2f0 con 0x7fa63c103fe0 2024-09-05T02:08:10.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.643+0000 7fa639ffb640 1 -- 172.21.15.103:0/870388903 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+12005 (secure 0 0 0) 0x7fa6300bc670 con 0x7fa63c103fe0 2024-09-05T02:08:10.645 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:08:10.646 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":35,"btime":"2024-09-05T02:06:39:264756+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:06:38.694889+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:08:10.646 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 35 2024-09-05T02:08:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:10 smithi139 ceph-mon[34262]: pgmap v522: 129 pgs: 129 active+clean; 344 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-09-05T02:08:10.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa62c0a2c00 msgr2=0x7fa62c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:10.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa62c0a2c00 0x7fa62c0a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6280095f0 tx=0x7fa628005f90 comp rx=0 tx=0).stop 2024-09-05T02:08:10.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c103fe0 msgr2=0x7fa63c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:10.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c103fe0 0x7fa63c19a500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa630005940 tx=0x7fa6300040c0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 shutdown_connections 2024-09-05T02:08:10.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63c109110 0x7fa63c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa63c104990 0x7fa63c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa62c0a2c00 0x7fa62c0a50c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 --2- 172.21.15.103:0/870388903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c103fe0 0x7fa63c19a500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:10.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 >> 172.21.15.103:0/870388903 conn(0x7fa63c0ffa60 msgr2=0x7fa63c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:10.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 shutdown_connections 2024-09-05T02:08:10.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:10.646+0000 7fa642fed640 1 -- 172.21.15.103:0/870388903 wait complete. 2024-09-05T02:08:11.461 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 154.3 --await-for 45.0' 2024-09-05T02:08:11.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 154.3 --await-for 45.0 2024-09-05T02:08:11.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:11 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/870388903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:08:11.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:11 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/870388903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:08:11.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:11 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/870388903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:08:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:08:12.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.341+0000 7f3402f4a640 1 -- 172.21.15.103:0/581442004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc105de0 msgr2=0x7f33fc10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:12.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.341+0000 7f3402f4a640 1 --2- 172.21.15.103:0/581442004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc105de0 0x7f33fc10d960 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009ca0 tx=0x7f33f402a8c0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.342+0000 7f3402f4a640 1 -- 172.21.15.103:0/581442004 shutdown_connections 2024-09-05T02:08:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.342+0000 7f3402f4a640 1 --2- 172.21.15.103:0/581442004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc105de0 0x7f33fc10d960 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.342+0000 7f3402f4a640 1 --2- 172.21.15.103:0/581442004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33fc101e20 0x7f33fc1056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.342+0000 7f3402f4a640 1 --2- 172.21.15.103:0/581442004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc101470 0x7f33fc101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.342+0000 7f3402f4a640 1 -- 172.21.15.103:0/581442004 >> 172.21.15.103:0/581442004 conn(0x7f33fc077230 msgr2=0x7f33fc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.342+0000 7f3402f4a640 1 -- 172.21.15.103:0/581442004 shutdown_connections 2024-09-05T02:08:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.342+0000 7f3402f4a640 1 -- 172.21.15.103:0/581442004 wait complete. 2024-09-05T02:08:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f3402f4a640 1 Processor -- start 2024-09-05T02:08:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f3402f4a640 1 -- start start 2024-09-05T02:08:12.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f3402f4a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc101470 0x7f33fc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f3402f4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc101e20 0x7f33fc19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f3402f4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33fc105de0 0x7f33fc19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f3402f4a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc110cf0 con 0x7f33fc101e20 2024-09-05T02:08:12.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f3402f4a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc110b70 con 0x7f33fc105de0 2024-09-05T02:08:12.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f3402f4a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc110e70 con 0x7f33fc101470 2024-09-05T02:08:12.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc101e20 0x7f33fc19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc101e20 0x7f33fc19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38494/0 (socket says 172.21.15.103:38494) 2024-09-05T02:08:12.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f33fbfff640 1 -- 172.21.15.103:0/31225445 learned_addr learned my addr 172.21.15.103:0/31225445 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f3400ce6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc101470 0x7f33fc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.343+0000 7f34014e7640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33fc105de0 0x7f33fc19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f33fbfff640 1 -- 172.21.15.103:0/31225445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc101470 msgr2=0x7f33fc19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f33fbfff640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc101470 0x7f33fc19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f33fbfff640 1 -- 172.21.15.103:0/31225445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33fc105de0 msgr2=0x7f33fc19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f33fbfff640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33fc105de0 0x7f33fc19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f33fbfff640 1 -- 172.21.15.103:0/31225445 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc19f500 con 0x7f33fc101e20 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f33fbfff640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc101e20 0x7f33fc19aca0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f33e800d950 tx=0x7f33e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f33f9ffb640 1 -- 172.21.15.103:0/31225445 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8017070 con 0x7f33fc101e20 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f33f9ffb640 1 -- 172.21.15.103:0/31225445 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33e8009c30 con 0x7f33fc101e20 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f33f9ffb640 1 -- 172.21.15.103:0/31225445 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e800f620 con 0x7f33fc101e20 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.344+0000 7f3402f4a640 1 -- 172.21.15.103:0/31225445 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc19f7f0 con 0x7f33fc101e20 2024-09-05T02:08:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.345+0000 7f3402f4a640 1 -- 172.21.15.103:0/31225445 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc1a70d0 con 0x7f33fc101e20 2024-09-05T02:08:12.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f3405323740 1 Processor -- start 2024-09-05T02:08:12.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f33f9ffb640 1 -- 172.21.15.103:0/31225445 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f33e80109a0 con 0x7f33fc101e20 2024-09-05T02:08:12.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f3405323740 1 -- start start 2024-09-05T02:08:12.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f3405323740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f3405323740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76fa0 0x555579e89c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f3405323740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e8a150 0x555579e8c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f3405323740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555579ddc2a0 con 0x555579e76680 2024-09-05T02:08:12.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f3405323740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555579d83b10 con 0x555579e76fa0 2024-09-05T02:08:12.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f3405323740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555579dd8260 con 0x555579e8a150 2024-09-05T02:08:12.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f34014e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f3400ce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76fa0 0x555579e89c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f34014e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e76a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38498/0 (socket says 172.21.15.103:38498) 2024-09-05T02:08:12.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f34014e7640 1 -- 172.21.15.103:0/771417144 learned_addr learned my addr 172.21.15.103:0/771417144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.348+0000 7f33fbfff640 1 --2- 172.21.15.103:0/771417144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e8a150 0x555579e8c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f33f9ffb640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f33ec0a2c00 0x7f33ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f34014e7640 1 -- 172.21.15.103:0/771417144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e8a150 msgr2=0x555579e8c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f34014e7640 1 --2- 172.21.15.103:0/771417144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e8a150 0x555579e8c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f34014e7640 1 -- 172.21.15.103:0/771417144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76fa0 msgr2=0x555579e89c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f34014e7640 1 --2- 172.21.15.103:0/771417144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76fa0 0x555579e89c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f34014e7640 1 -- 172.21.15.103:0/771417144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555579b4db90 con 0x555579e76680 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f33fbfff640 1 --2- 172.21.15.103:0/771417144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e8a150 0x555579e8c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f33f9ffb640 1 -- 172.21.15.103:0/31225445 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f33e80ceed0 con 0x7f33fc101e20 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f34014e7640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f33ec0a2c00 0x7f33ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f34014e7640 1 --2- 172.21.15.103:0/771417144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e76a60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009ca0 tx=0x7f33f4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15945ad4275d1511 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f33e6ffd640 1 -- 172.21.15.103:0/771417144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4031450 con 0x555579e76680 2024-09-05T02:08:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.349+0000 7f33e6ffd640 1 -- 172.21.15.103:0/771417144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33f4036030 con 0x555579e76680 2024-09-05T02:08:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f33e6ffd640 1 -- 172.21.15.103:0/771417144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f40046f0 con 0x555579e76680 2024-09-05T02:08:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f3405323740 1 -- 172.21.15.103:0/771417144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 msgr2=0x555579e76a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f3405323740 1 --2- 172.21.15.103:0/771417144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e76a60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009ca0 tx=0x7f33f4009a20 comp rx=0 tx=0).stop 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f34014e7640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f33ec0a2c00 0x7f33ec0a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f33f404a000 tx=0x7f33f40305c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f3405323740 1 -- 172.21.15.103:0/771417144 shutdown_connections 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f3405323740 1 --2- 172.21.15.103:0/771417144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e8a150 0x555579e8c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f3405323740 1 --2- 172.21.15.103:0/771417144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76fa0 0x555579e89c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f3405323740 1 --2- 172.21.15.103:0/771417144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e76a60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f3405323740 1 -- 172.21.15.103:0/771417144 >> 172.21.15.103:0/771417144 conn(0x555579e72cf0 msgr2=0x555579e730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f3405323740 1 -- 172.21.15.103:0/771417144 shutdown_connections 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.350+0000 7f3405323740 1 -- 172.21.15.103:0/771417144 wait complete. 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.351+0000 7f3405323740 1 -- start start 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.351+0000 7f3405323740 1 Processor -- start 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.351+0000 7f3405323740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76680 0x555579e870b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f3405323740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76fa0 0x555579e875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f3405323740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 0x555579e87b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f3405323740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555579e8cb40 con 0x555579e76fa0 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f3405323740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555579d646e0 con 0x555579e8a150 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f3405323740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555579e8ccc0 con 0x555579e76680 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76fa0 0x555579e875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76fa0 0x555579e875f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38510/0 (socket says 172.21.15.103:38510) 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f3400ce6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76680 0x555579e870b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33fbfff640 1 -- 172.21.15.103:0/1896844439 learned_addr learned my addr 172.21.15.103:0/1896844439 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f3405323740 10 client.0 _ll_register_callbacks cb 0x555579d88a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f34014e7640 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 0x555579e87b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33fbfff640 1 -- 172.21.15.103:0/1896844439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76680 msgr2=0x555579e870b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33fbfff640 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76680 0x555579e870b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33fbfff640 1 -- 172.21.15.103:0/1896844439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 msgr2=0x555579e87b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33fbfff640 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 0x555579e87b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33fbfff640 1 -- 172.21.15.103:0/1896844439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555579e960c0 con 0x555579e76fa0 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f3400ce6640 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76680 0x555579e870b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33fbfff640 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76fa0 0x555579e875f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f33e8000c00 tx=0x7f33e809b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2787d929f3a8fb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.352+0000 7f33dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f33dbfff640 1 -- 172.21.15.103:0/1896844439 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8020050 con 0x555579e76fa0 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f33dbfff640 1 -- 172.21.15.103:0/1896844439 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f33e809a880 con 0x555579e76fa0 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f3405323740 1 -- 172.21.15.103:0/1896844439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f4036030 con 0x555579e76fa0 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f33dbfff640 1 -- 172.21.15.103:0/1896844439 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f33e80b4050 con 0x555579e76fa0 2024-09-05T02:08:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f3405323740 10 client.16080 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f3405323740 1 -- 172.21.15.103:0/1896844439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555579e98a30 con 0x555579e76fa0 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f33dbfff640 1 -- 172.21.15.103:0/1896844439 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33e8009de0 con 0x555579e76fa0 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f33dbfff640 1 -- 172.21.15.103:0/1896844439 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f33e80b41f0 con 0x555579e76fa0 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f3405323740 10 client.16080 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f3405323740 10 client.16080 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f3405323740 20 client.16080 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f3405323740 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555579e99ef0 0x555579e9c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f3405323740 4 client.16080 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.353+0000 7f3405323740 1 -- 172.21.15.103:0/1896844439 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f33f4036030 con 0x555579e99ef0 2024-09-05T02:08:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.354+0000 7f3400ce6640 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555579e99ef0 0x555579e9c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.354+0000 7f33dbfff640 10 client.16080 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.354+0000 7f3400ce6640 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555579e99ef0 0x555579e9c2e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48feee560da83855 server_cookie=65567f608e30e396 in_seq=0 out_seq=0 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.355+0000 7f33dbfff640 1 -- 172.21.15.103:0/1896844439 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f33f4036030 con 0x555579e99ef0 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.355+0000 7f33dbfff640 10 client.16080 handle_command_reply: tid=0 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.355+0000 7f3405323740 1 client.16080 shutdown 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.355+0000 7f3405323740 1 -- 172.21.15.103:0/1896844439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76fa0 msgr2=0x555579e875f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.355+0000 7f3405323740 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76fa0 0x555579e875f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f33e8000c00 tx=0x7f33e809b200 comp rx=0 tx=0).stop 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f3405323740 1 -- 172.21.15.103:0/1896844439 shutdown_connections 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f3405323740 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555579e99ef0 0x555579e9c2e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f33dbfff640 0 client.16080 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f3405323740 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 0x555579e87b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f3405323740 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76fa0 0x555579e875f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f3405323740 1 --2- 172.21.15.103:0/1896844439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76680 0x555579e870b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f3405323740 1 -- 172.21.15.103:0/1896844439 >> 172.21.15.103:0/1896844439 conn(0x555579e72cf0 msgr2=0x555579e78980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:12.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f3405323740 1 -- 172.21.15.103:0/1896844439 shutdown_connections 2024-09-05T02:08:12.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f3405323740 1 -- 172.21.15.103:0/1896844439 wait complete. 2024-09-05T02:08:12.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.356+0000 7f3405323740 20 client.16080 trim_cache size 0 max 16384 2024-09-05T02:08:12.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.379+0000 7f3405323740 1 Processor -- start 2024-09-05T02:08:12.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f3405323740 1 -- start start 2024-09-05T02:08:12.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f3405323740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76680 0x555579e71d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f3405323740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579ea1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f3405323740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e8a150 0x555579ea2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f3405323740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1aca80 con 0x555579e8a150 2024-09-05T02:08:12.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f3405323740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1ae680 con 0x555579e76680 2024-09-05T02:08:12.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f3405323740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1ad150 con 0x555579e76fa0 2024-09-05T02:08:12.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f3400ce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76680 0x555579e71d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f34014e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e8a150 0x555579ea2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579ea1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579ea1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:41698/0 (socket says 172.21.15.103:41698) 2024-09-05T02:08:12.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f34014e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e8a150 0x555579ea2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38522/0 (socket says 172.21.15.103:38522) 2024-09-05T02:08:12.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f33fbfff640 1 -- 172.21.15.103:0/4189354304 learned_addr learned my addr 172.21.15.103:0/4189354304 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:12.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f33fbfff640 1 -- 172.21.15.103:0/4189354304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76680 msgr2=0x555579e71d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:12.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f33fbfff640 1 --2- 172.21.15.103:0/4189354304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76680 0x555579e71d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f33fbfff640 1 -- 172.21.15.103:0/4189354304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e8a150 msgr2=0x555579ea2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:12.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f33fbfff640 1 --2- 172.21.15.103:0/4189354304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e8a150 0x555579ea2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.380+0000 7f33fbfff640 1 -- 172.21.15.103:0/4189354304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555579e86060 con 0x555579e76fa0 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.381+0000 7f33fbfff640 1 --2- 172.21.15.103:0/4189354304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579ea1b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f33e809b3c0 tx=0x7f33e8099940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fef4f99f0ec25f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.381+0000 7f33e67fc640 1 -- 172.21.15.103:0/4189354304 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8020070 con 0x555579e76fa0 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.381+0000 7f33e67fc640 1 -- 172.21.15.103:0/4189354304 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33e80a3ff0 con 0x555579e76fa0 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.381+0000 7f33e67fc640 1 -- 172.21.15.103:0/4189354304 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e80b4380 con 0x555579e76fa0 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.382+0000 7f3405323740 1 -- 172.21.15.103:0/4189354304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 msgr2=0x555579ea1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.382+0000 7f3405323740 1 --2- 172.21.15.103:0/4189354304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579ea1b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f33e809b3c0 tx=0x7f33e8099940 comp rx=0 tx=0).stop 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.382+0000 7f3405323740 1 -- 172.21.15.103:0/4189354304 shutdown_connections 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.382+0000 7f3405323740 1 --2- 172.21.15.103:0/4189354304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e8a150 0x555579ea2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.382+0000 7f3405323740 1 --2- 172.21.15.103:0/4189354304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579ea1b40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.382+0000 7f3405323740 1 --2- 172.21.15.103:0/4189354304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e76680 0x555579e71d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.382+0000 7f3405323740 1 -- 172.21.15.103:0/4189354304 >> 172.21.15.103:0/4189354304 conn(0x555579e72cf0 msgr2=0x555579e71650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:12.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.382+0000 7f3405323740 1 -- 172.21.15.103:0/4189354304 shutdown_connections 2024-09-05T02:08:12.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.382+0000 7f3405323740 1 -- 172.21.15.103:0/4189354304 wait complete. 2024-09-05T02:08:12.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.383+0000 7f3405323740 1 -- start start 2024-09-05T02:08:12.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.383+0000 7f3405323740 1 Processor -- start 2024-09-05T02:08:12.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.383+0000 7f3405323740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e8e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.383+0000 7f3405323740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579e8ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3400ce6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e8e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3400ce6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e8e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38536/0 (socket says 172.21.15.103:38536) 2024-09-05T02:08:12.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3400ce6640 1 -- 172.21.15.103:0/3523264523 learned_addr learned my addr 172.21.15.103:0/3523264523 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:12.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3405323740 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 0x555579efdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1adfb0 con 0x555579e76680 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1ad820 con 0x555579e8a150 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1aaa80 con 0x555579e76fa0 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3405323740 10 client.0 _ll_register_callbacks cb 0x7f33e80d26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f33fbfff640 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579e8ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f34014e7640 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 0x555579efdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3400ce6640 1 -- 172.21.15.103:0/3523264523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 msgr2=0x555579e8ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3400ce6640 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579e8ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3400ce6640 1 -- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 msgr2=0x555579efdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3400ce6640 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 0x555579efdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3400ce6640 1 -- 172.21.15.103:0/3523264523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555579e76dd0 con 0x555579e76680 2024-09-05T02:08:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f33fbfff640 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579e8ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f34014e7640 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 0x555579efdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f3400ce6640 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e8e500 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f33f0012590 tx=0x7f33f0012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1accb94d8200960b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f33dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.384+0000 7f33dbfff640 1 -- 172.21.15.103:0/3523264523 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0005020 con 0x555579e76680 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e80a3ff0 con 0x555579e76680 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f33dbfff640 1 -- 172.21.15.103:0/3523264523 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f33f00051c0 con 0x555579e76680 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f33dbfff640 1 -- 172.21.15.103:0/3523264523 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f33f0033050 con 0x555579e76680 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f3405323740 10 client.16086 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555579f052d0 con 0x555579e76680 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f33dbfff640 1 -- 172.21.15.103:0/3523264523 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33e80a3ff0 con 0x555579e76680 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f33dbfff640 1 -- 172.21.15.103:0/3523264523 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f33f00047a0 con 0x555579e76680 2024-09-05T02:08:12.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f3405323740 10 client.16086 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:12.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f3405323740 10 client.16086 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:08:12.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.385+0000 7f3405323740 20 client.16086 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:12.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.386+0000 7f3405323740 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555579e99ef0 0x555579f07b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:12.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.386+0000 7f3405323740 4 client.16086 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.3, "await_for": 45.0}] 2024-09-05T02:08:12.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.386+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.3, "await_for": 45.0}) -- 0x555579de99d0 con 0x555579e99ef0 2024-09-05T02:08:12.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.386+0000 7f33fbfff640 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555579e99ef0 0x555579f07b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:12.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.386+0000 7f33dbfff640 10 client.16086 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:12.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:12.386+0000 7f33fbfff640 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555579e99ef0 0x555579f07b60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1709386a60bdfb3 server_cookie=aeecc553396d439c in_seq=0 out_seq=0 2024-09-05T02:08:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:12 smithi167 ceph-mon[35122]: pgmap v523: 129 pgs: 129 active+clean; 346 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-05T02:08:12.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:12 smithi139 ceph-mon[34262]: pgmap v523: 129 pgs: 129 active+clean; 346 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-05T02:08:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:12 smithi103 ceph-mon[30902]: pgmap v523: 129 pgs: 129 active+clean; 346 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-05T02:08:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f33dbfff640 1 -- 172.21.15.103:0/3523264523 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555579de99d0 con 0x555579e99ef0 2024-09-05T02:08:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f33dbfff640 10 client.16086 handle_command_reply: tid=0 2024-09-05T02:08:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 client.16086 shutdown 2024-09-05T02:08:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 msgr2=0x555579e8e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e8e500 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f33f0012590 tx=0x7f33f0012b10 comp rx=0 tx=0).stop 2024-09-05T02:08:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 shutdown_connections 2024-09-05T02:08:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x555579e99ef0 0x555579f07b60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555579e8a150 0x555579efdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555579e76fa0 0x555579e8ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f33dbfff640 0 client.16086 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 --2- 172.21.15.103:0/3523264523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555579e76680 0x555579e8e500 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 >> 172.21.15.103:0/3523264523 conn(0x555579e72cf0 msgr2=0x555579e87890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 shutdown_connections 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.859+0000 7f3405323740 1 -- 172.21.15.103:0/3523264523 wait complete. 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.860+0000 7f3405323740 20 client.16086 trim_cache size 0 max 16384 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:08:13.863 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 35, 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 18, 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "8e1959f2": { 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 18, 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 154.3, 2024-09-05T02:08:13.864 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:13.865 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:08:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.860+0000 7f3402f4a640 1 -- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f33ec0a2c00 msgr2=0x7f33ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.860+0000 7f3402f4a640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f33ec0a2c00 0x7f33ec0a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f33f404a000 tx=0x7f33f40305c0 comp rx=0 tx=0).stop 2024-09-05T02:08:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.860+0000 7f3402f4a640 1 -- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc101e20 msgr2=0x7f33fc19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.860+0000 7f3402f4a640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc101e20 0x7f33fc19aca0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f33e800d950 tx=0x7f33e800de10 comp rx=0 tx=0).stop 2024-09-05T02:08:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.861+0000 7f3402f4a640 1 -- 172.21.15.103:0/31225445 shutdown_connections 2024-09-05T02:08:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.861+0000 7f3402f4a640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33fc105de0 0x7f33fc19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.861+0000 7f3402f4a640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33fc101e20 0x7f33fc19aca0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.861+0000 7f3402f4a640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f33ec0a2c00 0x7f33ec0a50c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.861+0000 7f3402f4a640 1 --2- 172.21.15.103:0/31225445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33fc101470 0x7f33fc19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.861+0000 7f3402f4a640 1 -- 172.21.15.103:0/31225445 >> 172.21.15.103:0/31225445 conn(0x7f33fc077230 msgr2=0x7f33fc103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.861+0000 7f3402f4a640 1 -- 172.21.15.103:0/31225445 shutdown_connections 2024-09-05T02:08:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:13.861+0000 7f3402f4a640 1 -- 172.21.15.103:0/31225445 wait complete. 2024-09-05T02:08:13.875 INFO:tasks.workunit.client.0.smithi103.stdout: 59 11431 23 7191 60 3674 81 2024-09-05T02:08:14.586 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:14 smithi103 ceph-mon[30902]: pgmap v524: 129 pgs: 129 active+clean; 350 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-09-05T02:08:14.620 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8e1959f2', quiesce took 5.2 seconds. Will release after: 29.099999999999998 2024-09-05T02:08:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:14 smithi167 ceph-mon[35122]: pgmap v524: 129 pgs: 129 active+clean; 350 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-09-05T02:08:14.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:14 smithi139 ceph-mon[34262]: pgmap v524: 129 pgs: 129 active+clean; 350 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-09-05T02:08:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.966+0000 7fd492f11640 1 -- 172.21.15.103:0/1744457224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c100a50 msgr2=0x7fd48c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.966+0000 7fd492f11640 1 --2- 172.21.15.103:0/1744457224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c100a50 0x7fd48c100e50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd480009860 tx=0x7fd48002a340 comp rx=0 tx=0).stop 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.967+0000 7fd492f11640 1 -- 172.21.15.103:0/1744457224 shutdown_connections 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.967+0000 7fd492f11640 1 --2- 172.21.15.103:0/1744457224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48c102600 0x7fd48c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.967+0000 7fd492f11640 1 --2- 172.21.15.103:0/1744457224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd48c101c40 0x7fd48c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.967+0000 7fd492f11640 1 --2- 172.21.15.103:0/1744457224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c100a50 0x7fd48c100e50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.967+0000 7fd492f11640 1 -- 172.21.15.103:0/1744457224 >> 172.21.15.103:0/1744457224 conn(0x7fd48c0fc1d0 msgr2=0x7fd48c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.967+0000 7fd492f11640 1 -- 172.21.15.103:0/1744457224 shutdown_connections 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.967+0000 7fd492f11640 1 -- 172.21.15.103:0/1744457224 wait complete. 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.967+0000 7fd492f11640 1 Processor -- start 2024-09-05T02:08:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.967+0000 7fd492f11640 1 -- start start 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd492f11640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd48c100a50 0x7fd48c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd492f11640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c101c40 0x7fd48c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd492f11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48c102600 0x7fd48c19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd492f11640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10d6d0 con 0x7fd48c102600 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd492f11640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10d550 con 0x7fd48c101c40 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd492f11640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10d850 con 0x7fd48c100a50 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd490cad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd48c100a50 0x7fd48c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd490cad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd48c100a50 0x7fd48c197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:41722/0 (socket says 172.21.15.103:41722) 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd4914ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48c102600 0x7fd48c19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd490cad640 1 -- 172.21.15.103:0/2662707994 learned_addr learned my addr 172.21.15.103:0/2662707994 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd48bfff640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c101c40 0x7fd48c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd490cad640 1 -- 172.21.15.103:0/2662707994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c101c40 msgr2=0x7fd48c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd490cad640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c101c40 0x7fd48c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd490cad640 1 -- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48c102600 msgr2=0x7fd48c19f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd490cad640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48c102600 0x7fd48c19f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd490cad640 1 -- 172.21.15.103:0/2662707994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c19f970 con 0x7fd48c100a50 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd4914ae640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48c102600 0x7fd48c19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.968+0000 7fd490cad640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd48c100a50 0x7fd48c197d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd480005940 tx=0x7fd480030e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.969+0000 7fd489ffb640 1 -- 172.21.15.103:0/2662707994 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480037050 con 0x7fd48c100a50 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.969+0000 7fd489ffb640 1 -- 172.21.15.103:0/2662707994 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd480043070 con 0x7fd48c100a50 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.969+0000 7fd489ffb640 1 -- 172.21.15.103:0/2662707994 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480003c70 con 0x7fd48c100a50 2024-09-05T02:08:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.969+0000 7fd492f11640 1 -- 172.21.15.103:0/2662707994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c109850 con 0x7fd48c100a50 2024-09-05T02:08:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.969+0000 7fd492f11640 1 -- 172.21.15.103:0/2662707994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c109cf0 con 0x7fd48c100a50 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.972+0000 7fd489ffb640 1 -- 172.21.15.103:0/2662707994 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd480004040 con 0x7fd48c100a50 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.972+0000 7fd495ca5740 1 Processor -- start 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.972+0000 7fd495ca5740 1 -- start start 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.972+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.972+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 0x5600c64a8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.972+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.972+0000 7fd495ca5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600c64072f0 con 0x5600c64a39a0 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.972+0000 7fd495ca5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5600c63af270 con 0x5600c64a9370 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.972+0000 7fd495ca5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5600c64032b0 con 0x5600c64a42e0 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd4914ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd490cad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 0x5600c64a8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd4914ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38568/0 (socket says 172.21.15.103:38568) 2024-09-05T02:08:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd4914ae640 1 -- 172.21.15.103:0/1551212031 learned_addr learned my addr 172.21.15.103:0/1551212031 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd48bfff640 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd489ffb640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd47c0a2cb0 0x7fd47c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd490cad640 1 -- 172.21.15.103:0/1551212031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 msgr2=0x5600c64ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd490cad640 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd490cad640 1 -- 172.21.15.103:0/1551212031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 msgr2=0x5600c64a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd490cad640 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd490cad640 1 -- 172.21.15.103:0/1551212031 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c63ae340 con 0x5600c64a42e0 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd48bfff640 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd4914ae640 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd4914ae640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd47c0a2cb0 0x7fd47c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd489ffb640 1 -- 172.21.15.103:0/2662707994 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd4800ef800 con 0x7fd48c100a50 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd490cad640 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 0x5600c64a8da0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd4800ecf20 tx=0x7fd48004b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90045356d7e7fb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.973+0000 7fd476ffd640 1 -- 172.21.15.103:0/1551212031 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4800bb7f0 con 0x5600c64a42e0 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd476ffd640 1 -- 172.21.15.103:0/1551212031 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd480038040 con 0x5600c64a42e0 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd476ffd640 1 -- 172.21.15.103:0/1551212031 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4800bb610 con 0x5600c64a42e0 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd495ca5740 1 -- 172.21.15.103:0/1551212031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 msgr2=0x5600c64a8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd4914ae640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd47c0a2cb0 0x7fd47c0a5170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd484011c00 tx=0x7fd484004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 0x5600c64a8da0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd4800ecf20 tx=0x7fd48004b000 comp rx=0 tx=0).stop 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd495ca5740 1 -- 172.21.15.103:0/1551212031 shutdown_connections 2024-09-05T02:08:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 0x5600c64a8da0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1551212031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.974+0000 7fd495ca5740 1 -- 172.21.15.103:0/1551212031 >> 172.21.15.103:0/1551212031 conn(0x5600c649f010 msgr2=0x5600c649f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.975+0000 7fd495ca5740 1 -- 172.21.15.103:0/1551212031 shutdown_connections 2024-09-05T02:08:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.975+0000 7fd495ca5740 1 -- 172.21.15.103:0/1551212031 wait complete. 2024-09-05T02:08:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.975+0000 7fd495ca5740 1 -- start start 2024-09-05T02:08:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.975+0000 7fd495ca5740 1 Processor -- start 2024-09-05T02:08:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.975+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.975+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 0x5600c64adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.975+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.975+0000 7fd495ca5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600c64032b0 con 0x5600c64a39a0 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd495ca5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5600c6390290 con 0x5600c64a9370 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd495ca5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5600c64ac100 con 0x5600c64a42e0 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd490cad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd495ca5740 10 client.0 _ll_register_callbacks cb 0x5600c63c1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd490cad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38580/0 (socket says 172.21.15.103:38580) 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd490cad640 1 -- 172.21.15.103:0/3647587438 learned_addr learned my addr 172.21.15.103:0/3647587438 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd48bfff640 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 0x5600c64adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd4914ae640 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd490cad640 1 -- 172.21.15.103:0/3647587438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 msgr2=0x5600c64adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd490cad640 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 0x5600c64adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd490cad640 1 -- 172.21.15.103:0/3647587438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 msgr2=0x5600c64b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd490cad640 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd490cad640 1 -- 172.21.15.103:0/3647587438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c611a210 con 0x5600c64a39a0 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd490cad640 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64ad7b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd4800ed2c0 tx=0x7fd480004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f1fd40b88ef218 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd46bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd46bfff640 1 -- 172.21.15.103:0/3647587438 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4800bb610 con 0x5600c64a39a0 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd46bfff640 1 -- 172.21.15.103:0/3647587438 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd4800b9d50 con 0x5600c64a39a0 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd495ca5740 1 -- 172.21.15.103:0/3647587438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd480038040 con 0x5600c64a39a0 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd46bfff640 1 -- 172.21.15.103:0/3647587438 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd4800bb7f0 con 0x5600c64a39a0 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.976+0000 7fd495ca5740 10 client.16098 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd495ca5740 1 -- 172.21.15.103:0/3647587438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5600c61e6620 con 0x5600c64a39a0 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd46bfff640 1 -- 172.21.15.103:0/3647587438 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd480038040 con 0x5600c64a39a0 2024-09-05T02:08:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd46bfff640 1 -- 172.21.15.103:0/3647587438 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fd480045310 con 0x5600c64a39a0 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd495ca5740 10 client.16098 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd495ca5740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd495ca5740 20 client.16098 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5600c64b9300 0x5600c64bb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd495ca5740 4 client.16098 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd495ca5740 1 -- 172.21.15.103:0/3647587438 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600c6414a20 con 0x5600c64b9300 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.977+0000 7fd48bfff640 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5600c64b9300 0x5600c64bb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.978+0000 7fd46bfff640 10 client.16098 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.978+0000 7fd48bfff640 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5600c64b9300 0x5600c64bb6f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bcdf22ecc7827e3 server_cookie=50a751776f0f0c5f in_seq=0 out_seq=0 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.978+0000 7fd46bfff640 1 -- 172.21.15.103:0/3647587438 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600c6414a20 con 0x5600c64b9300 2024-09-05T02:08:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.978+0000 7fd46bfff640 10 client.16098 handle_command_reply: tid=0 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.978+0000 7fd495ca5740 1 client.16098 shutdown 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 -- 172.21.15.103:0/3647587438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 msgr2=0x5600c64ad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64ad7b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd4800ed2c0 tx=0x7fd480004610 comp rx=0 tx=0).stop 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 -- 172.21.15.103:0/3647587438 shutdown_connections 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5600c64b9300 0x5600c64bb6f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a42e0 0x5600c64adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3647587438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a39a0 0x5600c64ad7b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 -- 172.21.15.103:0/3647587438 >> 172.21.15.103:0/3647587438 conn(0x5600c649f010 msgr2=0x5600c64a0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd46bfff640 0 client.16098 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 -- 172.21.15.103:0/3647587438 shutdown_connections 2024-09-05T02:08:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 1 -- 172.21.15.103:0/3647587438 wait complete. 2024-09-05T02:08:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.979+0000 7fd495ca5740 20 client.16098 trim_cache size 0 max 16384 2024-09-05T02:08:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd495ca5740 1 Processor -- start 2024-09-05T02:08:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd495ca5740 1 -- start start 2024-09-05T02:08:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a42e0 0x5600c64b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a9370 0x5600c64a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd495ca5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1a84a0 con 0x5600c64a9370 2024-09-05T02:08:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd495ca5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1aa0a0 con 0x5600c64a42e0 2024-09-05T02:08:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd495ca5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1a8b70 con 0x5600c64a39a0 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a42e0 0x5600c64b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd490cad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd4914ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a9370 0x5600c64a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a42e0 0x5600c64b4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:49714/0 (socket says 172.21.15.103:49714) 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:14.999+0000 7fd48bfff640 1 -- 172.21.15.103:0/3936813018 learned_addr learned my addr 172.21.15.103:0/3936813018 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.000+0000 7fd48bfff640 1 -- 172.21.15.103:0/3936813018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 msgr2=0x5600c64adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.000+0000 7fd48bfff640 1 --2- 172.21.15.103:0/3936813018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.000+0000 7fd48bfff640 1 -- 172.21.15.103:0/3936813018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a9370 msgr2=0x5600c64a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.000+0000 7fd48bfff640 1 --2- 172.21.15.103:0/3936813018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a9370 0x5600c64a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.000+0000 7fd48bfff640 1 -- 172.21.15.103:0/3936813018 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c64ac230 con 0x5600c64a42e0 2024-09-05T02:08:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.000+0000 7fd48bfff640 1 --2- 172.21.15.103:0/3936813018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a42e0 0x5600c64b4c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd478012010 tx=0x7fd478004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cec54a2cba24ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.000+0000 7fd4767fc640 1 -- 172.21.15.103:0/3936813018 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478004030 con 0x5600c64a42e0 2024-09-05T02:08:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.000+0000 7fd4767fc640 1 -- 172.21.15.103:0/3936813018 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd47801d640 con 0x5600c64a42e0 2024-09-05T02:08:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.000+0000 7fd4767fc640 1 -- 172.21.15.103:0/3936813018 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4780038e0 con 0x5600c64a42e0 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.001+0000 7fd495ca5740 1 -- 172.21.15.103:0/3936813018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a42e0 msgr2=0x5600c64b4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.001+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3936813018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a42e0 0x5600c64b4c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd478012010 tx=0x7fd478004730 comp rx=0 tx=0).stop 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.001+0000 7fd495ca5740 1 -- 172.21.15.103:0/3936813018 shutdown_connections 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.001+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3936813018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a9370 0x5600c64a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.001+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3936813018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a42e0 0x5600c64b4c50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.001+0000 7fd495ca5740 1 --2- 172.21.15.103:0/3936813018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.001+0000 7fd495ca5740 1 -- 172.21.15.103:0/3936813018 >> 172.21.15.103:0/3936813018 conn(0x5600c649f010 msgr2=0x5600c64ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.001+0000 7fd495ca5740 1 -- 172.21.15.103:0/3936813018 shutdown_connections 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.001+0000 7fd495ca5740 1 -- 172.21.15.103:0/3936813018 wait complete. 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd495ca5740 1 -- start start 2024-09-05T02:08:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd495ca5740 1 Processor -- start 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a42e0 0x5600c64b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd495ca5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd495ca5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1a99d0 con 0x5600c64a42e0 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd495ca5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1a9240 con 0x5600c64a9370 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd495ca5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1a6480 con 0x5600c64a39a0 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd490cad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd490cad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:41762/0 (socket says 172.21.15.103:41762) 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd490cad640 1 -- 172.21.15.103:0/1284740453 learned_addr learned my addr 172.21.15.103:0/1284740453 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.002+0000 7fd48bfff640 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a42e0 0x5600c64b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd495ca5740 10 client.0 _ll_register_callbacks cb 0x7fd484009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd4914ae640 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd490cad640 1 -- 172.21.15.103:0/1284740453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 msgr2=0x5600c64b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd490cad640 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd490cad640 1 -- 172.21.15.103:0/1284740453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a42e0 msgr2=0x5600c64b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd490cad640 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a42e0 0x5600c64b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd490cad640 1 -- 172.21.15.103:0/1284740453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c64b3ed0 con 0x5600c64a39a0 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd490cad640 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64b2370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd4800ed2c0 tx=0x7fd48002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e1ecb0a76cf7e45 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd46bfff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd46bfff640 1 -- 172.21.15.103:0/1284740453 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4800bb610 con 0x5600c64a39a0 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd46bfff640 1 -- 172.21.15.103:0/1284740453 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd480004330 con 0x5600c64a39a0 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd495ca5740 1 -- 172.21.15.103:0/1284740453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd47801d640 con 0x5600c64a39a0 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd46bfff640 1 -- 172.21.15.103:0/1284740453 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd4800bba80 con 0x5600c64a39a0 2024-09-05T02:08:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd495ca5740 10 client.25465 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.003+0000 7fd495ca5740 1 -- 172.21.15.103:0/1284740453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5600c64b3ed0 con 0x5600c64a39a0 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd46bfff640 1 -- 172.21.15.103:0/1284740453 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd47801d640 con 0x5600c64a39a0 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd46bfff640 1 -- 172.21.15.103:0/1284740453 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fd4800f5170 con 0x5600c64a39a0 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd495ca5740 10 client.25465 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd495ca5740 10 client.25465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd495ca5740 20 client.25465 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5600c6524630 0x5600c6526a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd495ca5740 4 client.25465 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd495ca5740 1 -- 172.21.15.103:0/1284740453 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600c640ed70 con 0x5600c6524630 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd48bfff640 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5600c6524630 0x5600c6526a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd46bfff640 10 client.25465 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.004+0000 7fd48bfff640 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5600c6524630 0x5600c6526a50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acfb04def4104416 server_cookie=535d6b3666df3d5d in_seq=0 out_seq=0 2024-09-05T02:08:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd46bfff640 1 -- 172.21.15.103:0/1284740453 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600c640ed70 con 0x5600c6524630 2024-09-05T02:08:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd46bfff640 10 client.25465 handle_command_reply: tid=0 2024-09-05T02:08:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd495ca5740 1 client.25465 shutdown 2024-09-05T02:08:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd495ca5740 1 -- 172.21.15.103:0/1284740453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 msgr2=0x5600c64b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64b2370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd4800ed2c0 tx=0x7fd48002a340 comp rx=0 tx=0).stop 2024-09-05T02:08:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd495ca5740 1 -- 172.21.15.103:0/1284740453 shutdown_connections 2024-09-05T02:08:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5600c6524630 0x5600c6526a50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600c64a9370 0x5600c64b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c64a42e0 0x5600c64b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd495ca5740 1 --2- 172.21.15.103:0/1284740453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600c64a39a0 0x5600c64b2370 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.080+0000 7fd495ca5740 1 -- 172.21.15.103:0/1284740453 >> 172.21.15.103:0/1284740453 conn(0x5600c649f010 msgr2=0x5600c64ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.081+0000 7fd46bfff640 0 client.25465 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.081+0000 7fd495ca5740 1 -- 172.21.15.103:0/1284740453 shutdown_connections 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.081+0000 7fd495ca5740 1 -- 172.21.15.103:0/1284740453 wait complete. 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.081+0000 7fd495ca5740 20 client.25465 trim_cache size 0 max 16384 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:08:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:08:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:08:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 -- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd47c0a2cb0 msgr2=0x7fd47c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd47c0a2cb0 0x7fd47c0a5170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd484011c00 tx=0x7fd484004a40 comp rx=0 tx=0).stop 2024-09-05T02:08:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 -- 172.21.15.103:0/2662707994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd48c100a50 msgr2=0x7fd48c197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd48c100a50 0x7fd48c197d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd480005940 tx=0x7fd480030e00 comp rx=0 tx=0).stop 2024-09-05T02:08:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 -- 172.21.15.103:0/2662707994 shutdown_connections 2024-09-05T02:08:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48c102600 0x7fd48c19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c101c40 0x7fd48c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd47c0a2cb0 0x7fd47c0a5170 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 --2- 172.21.15.103:0/2662707994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd48c100a50 0x7fd48c197d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 -- 172.21.15.103:0/2662707994 >> 172.21.15.103:0/2662707994 conn(0x7fd48c0fc1d0 msgr2=0x7fd48c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 -- 172.21.15.103:0/2662707994 shutdown_connections 2024-09-05T02:08:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:15.082+0000 7fd492f11640 1 -- 172.21.15.103:0/2662707994 wait complete. 2024-09-05T02:08:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:16 smithi167 ceph-mon[35122]: pgmap v525: 129 pgs: 129 active+clean; 352 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-09-05T02:08:16.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:16 smithi139 ceph-mon[34262]: pgmap v525: 129 pgs: 129 active+clean; 352 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-09-05T02:08:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:16 smithi103 ceph-mon[30902]: pgmap v525: 129 pgs: 129 active+clean; 352 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-09-05T02:08:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:18 smithi167 ceph-mon[35122]: pgmap v526: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-09-05T02:08:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:18 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:08:18.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:18 smithi139 ceph-mon[34262]: pgmap v526: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-09-05T02:08:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:18 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:08:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:18 smithi103 ceph-mon[30902]: pgmap v526: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-09-05T02:08:18.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:18 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:08:19.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:08:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:08:19.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:08:19.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:08:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:08:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:08:19.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:08:19.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:08:19.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:08:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:20 smithi167 ceph-mon[35122]: pgmap v527: 129 pgs: 129 active+clean; 350 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-09-05T02:08:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:20 smithi139 ceph-mon[34262]: pgmap v527: 129 pgs: 129 active+clean; 350 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-09-05T02:08:20.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:20 smithi103 ceph-mon[30902]: pgmap v527: 129 pgs: 129 active+clean; 350 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-09-05T02:08:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:22 smithi167 ceph-mon[35122]: pgmap v528: 129 pgs: 129 active+clean; 350 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 927 KiB/s wr, 173 op/s 2024-09-05T02:08:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:22 smithi139 ceph-mon[34262]: pgmap v528: 129 pgs: 129 active+clean; 350 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 927 KiB/s wr, 173 op/s 2024-09-05T02:08:22.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:22 smithi103 ceph-mon[30902]: pgmap v528: 129 pgs: 129 active+clean; 350 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 927 KiB/s wr, 173 op/s 2024-09-05T02:08:23.875 INFO:tasks.workunit.client.0.smithi103.stdout: 59 12604 0 7704 0 4602 0 2024-09-05T02:08:24.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:24 smithi167 ceph-mon[35122]: pgmap v529: 129 pgs: 129 active+clean; 352 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 937 KiB/s wr, 161 op/s 2024-09-05T02:08:24.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:24 smithi139 ceph-mon[34262]: pgmap v529: 129 pgs: 129 active+clean; 352 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 937 KiB/s wr, 161 op/s 2024-09-05T02:08:24.775 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:24 smithi103 ceph-mon[30902]: pgmap v529: 129 pgs: 129 active+clean; 352 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 937 KiB/s wr, 161 op/s 2024-09-05T02:08:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:26 smithi167 ceph-mon[35122]: pgmap v530: 129 pgs: 129 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 631 KiB/s wr, 132 op/s 2024-09-05T02:08:26.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:26 smithi139 ceph-mon[34262]: pgmap v530: 129 pgs: 129 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 631 KiB/s wr, 132 op/s 2024-09-05T02:08:26.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:26 smithi103 ceph-mon[30902]: pgmap v530: 129 pgs: 129 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 631 KiB/s wr, 132 op/s 2024-09-05T02:08:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 -- 172.21.15.103:0/975819528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 msgr2=0x7feb98108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/975819528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 0x7feb98108ea0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7feb90009ca0 tx=0x7feb9002a8c0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 -- 172.21.15.103:0/975819528 shutdown_connections 2024-09-05T02:08:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/975819528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 0x7feb98108ea0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/975819528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb98101c50 0x7feb981020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/975819528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb98100a60 0x7feb98100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 -- 172.21.15.103:0/975819528 >> 172.21.15.103:0/975819528 conn(0x7feb980fc1e0 msgr2=0x7feb980fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 -- 172.21.15.103:0/975819528 shutdown_connections 2024-09-05T02:08:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 -- 172.21.15.103:0/975819528 wait complete. 2024-09-05T02:08:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 Processor -- start 2024-09-05T02:08:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.198+0000 7feb9e8b7640 1 -- start start 2024-09-05T02:08:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9e8b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb98100a60 0x7feb98196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9e8b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb98101c50 0x7feb981970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9e8b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 0x7feb9819e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9e8b7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb9810c590 con 0x7feb98102610 2024-09-05T02:08:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9e8b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb9810c410 con 0x7feb98101c50 2024-09-05T02:08:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9e8b7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb9810c710 con 0x7feb98100a60 2024-09-05T02:08:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9ce54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 0x7feb9819e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9ce54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 0x7feb9819e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37476/0 (socket says 172.21.15.103:37476) 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9ce54640 1 -- 172.21.15.103:0/2224312152 learned_addr learned my addr 172.21.15.103:0/2224312152 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb97fff640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb98100a60 0x7feb98196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb977fe640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb98101c50 0x7feb981970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9ce54640 1 -- 172.21.15.103:0/2224312152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb98100a60 msgr2=0x7feb98196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9ce54640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb98100a60 0x7feb98196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9ce54640 1 -- 172.21.15.103:0/2224312152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb98101c50 msgr2=0x7feb981970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9ce54640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb98101c50 0x7feb981970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb9ce54640 1 -- 172.21.15.103:0/2224312152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9819e7d0 con 0x7feb98102610 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.199+0000 7feb97fff640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb98100a60 0x7feb98196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.200+0000 7feb9ce54640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 0x7feb9819e160 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feb90031970 tx=0x7feb90003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.200+0000 7feb957fa640 1 -- 172.21.15.103:0/2224312152 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90003d90 con 0x7feb98102610 2024-09-05T02:08:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.200+0000 7feb957fa640 1 -- 172.21.15.103:0/2224312152 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb900044f0 con 0x7feb98102610 2024-09-05T02:08:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.200+0000 7feb9e8b7640 1 -- 172.21.15.103:0/2224312152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9819ea60 con 0x7feb98102610 2024-09-05T02:08:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.200+0000 7feb957fa640 1 -- 172.21.15.103:0/2224312152 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90030bd0 con 0x7feb98102610 2024-09-05T02:08:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.200+0000 7feb9e8b7640 1 -- 172.21.15.103:0/2224312152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9819efb0 con 0x7feb98102610 2024-09-05T02:08:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feb957fa640 1 -- 172.21.15.103:0/2224312152 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feb90004040 con 0x7feb98102610 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feba1653740 1 Processor -- start 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feba1653740 1 -- start start 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feba1653740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b41eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feba1653740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41f2e0 0x56010b423da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feba1653740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 0x56010b426830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feba1653740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56010b3822f0 con 0x56010b41e9a0 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feba1653740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56010b32a270 con 0x56010b424370 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feba1653740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56010b37e2b0 con 0x56010b41f2e0 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feb977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b41eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feb977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b41eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37478/0 (socket says 172.21.15.103:37478) 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feb977fe640 1 -- 172.21.15.103:0/360884264 learned_addr learned my addr 172.21.15.103:0/360884264 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feb97fff640 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41f2e0 0x56010b423da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feb957fa640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feb800a2c30 0x7feb800a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.203+0000 7feb977fe640 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 0x56010b426830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb977fe640 1 -- 172.21.15.103:0/360884264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41f2e0 msgr2=0x56010b423da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb977fe640 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41f2e0 0x56010b423da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb977fe640 1 -- 172.21.15.103:0/360884264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 msgr2=0x56010b426830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb9ce54640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feb800a2c30 0x7feb800a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb977fe640 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 0x56010b426830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb977fe640 1 -- 172.21.15.103:0/360884264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010b329340 con 0x56010b41e9a0 2024-09-05T02:08:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb97fff640 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41f2e0 0x56010b423da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb957fa640 1 -- 172.21.15.103:0/2224312152 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feb900f0860 con 0x7feb98102610 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb977fe640 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b41eda0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009390 tx=0x7feb8c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbc2667d60e2771 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb7e7fc640 1 -- 172.21.15.103:0/360884264 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c004030 con 0x56010b41e9a0 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb9ce54640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feb800a2c30 0x7feb800a50f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feb900b9290 tx=0x7feb90087b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb7e7fc640 1 -- 172.21.15.103:0/360884264 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8c0035f0 con 0x56010b41e9a0 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.204+0000 7feb7e7fc640 1 -- 172.21.15.103:0/360884264 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c010690 con 0x56010b41e9a0 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 -- 172.21.15.103:0/360884264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 msgr2=0x56010b41eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b41eda0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009390 tx=0x7feb8c009850 comp rx=0 tx=0).stop 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 -- 172.21.15.103:0/360884264 shutdown_connections 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 0x56010b426830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41f2e0 0x56010b423da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 --2- 172.21.15.103:0/360884264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b41eda0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 -- 172.21.15.103:0/360884264 >> 172.21.15.103:0/360884264 conn(0x56010b41a010 msgr2=0x56010b41a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 -- 172.21.15.103:0/360884264 shutdown_connections 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 -- 172.21.15.103:0/360884264 wait complete. 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 -- start start 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.205+0000 7feba1653740 1 Processor -- start 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feba1653740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b428b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feba1653740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 0x56010b4290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feba1653740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b424370 0x56010b430110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feba1653740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56010b4272e0 con 0x56010b41e9a0 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feba1653740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56010b30b290 con 0x56010b41f2e0 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feba1653740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56010b427460 con 0x56010b424370 2024-09-05T02:08:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feba1653740 10 client.0 _ll_register_callbacks cb 0x56010b33c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b428b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b428b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37492/0 (socket says 172.21.15.103:37492) 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 -- 172.21.15.103:0/136637497 learned_addr learned my addr 172.21.15.103:0/136637497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b424370 0x56010b430110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb97fff640 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 0x56010b4290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 -- 172.21.15.103:0/136637497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b424370 msgr2=0x56010b430110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b424370 0x56010b430110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 -- 172.21.15.103:0/136637497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 msgr2=0x56010b4290d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 0x56010b4290d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 -- 172.21.15.103:0/136637497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56010b095210 con 0x56010b41e9a0 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb977fe640 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b428b90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7feb8c005c20 tx=0x7feb8c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439ef5bf63b5122b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb777fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:08:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb777fe640 1 -- 172.21.15.103:0/136637497 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c02e060 con 0x56010b41e9a0 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feb777fe640 1 -- 172.21.15.103:0/136637497 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feb8c027460 con 0x56010b41e9a0 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.206+0000 7feba1653740 1 -- 172.21.15.103:0/136637497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8c0035f0 con 0x56010b41e9a0 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feb777fe640 1 -- 172.21.15.103:0/136637497 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7feb8c035050 con 0x56010b41e9a0 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feba1653740 10 client.16116 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feba1653740 1 -- 172.21.15.103:0/136637497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56010b161620 con 0x56010b41e9a0 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feb777fe640 1 -- 172.21.15.103:0/136637497 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8c0035f0 con 0x56010b41e9a0 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feb777fe640 1 -- 172.21.15.103:0/136637497 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7feb8c0297d0 con 0x56010b41e9a0 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feba1653740 10 client.16116 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feba1653740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feba1653740 20 client.16116 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feba1653740 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56010b434580 0x56010b436970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feba1653740 4 client.16116 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:08:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feba1653740 1 -- 172.21.15.103:0/136637497 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56010b38fa20 con 0x56010b434580 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.207+0000 7feb97fff640 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56010b434580 0x56010b436970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feb777fe640 10 client.16116 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feb97fff640 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56010b434580 0x56010b436970 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9af2599cce1782c server_cookie=429b096f16a5d201 in_seq=0 out_seq=0 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feb777fe640 1 -- 172.21.15.103:0/136637497 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56010b38fa20 con 0x56010b434580 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feb777fe640 10 client.16116 handle_command_reply: tid=0 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feba1653740 1 client.16116 shutdown 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feba1653740 1 -- 172.21.15.103:0/136637497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 msgr2=0x56010b428b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feba1653740 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b428b90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7feb8c005c20 tx=0x7feb8c003d40 comp rx=0 tx=0).stop 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feba1653740 1 -- 172.21.15.103:0/136637497 shutdown_connections 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feba1653740 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56010b434580 0x56010b436970 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feba1653740 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b424370 0x56010b430110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feb777fe640 0 client.16116 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feba1653740 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 0x56010b4290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feba1653740 1 --2- 172.21.15.103:0/136637497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41e9a0 0x56010b428b90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.208+0000 7feba1653740 1 -- 172.21.15.103:0/136637497 >> 172.21.15.103:0/136637497 conn(0x56010b41a010 msgr2=0x56010b41b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.209+0000 7feba1653740 1 -- 172.21.15.103:0/136637497 shutdown_connections 2024-09-05T02:08:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.209+0000 7feba1653740 1 -- 172.21.15.103:0/136637497 wait complete. 2024-09-05T02:08:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.209+0000 7feba1653740 20 client.16116 trim_cache size 0 max 16384 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feba1653740 1 Processor -- start 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feba1653740 1 -- start start 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feba1653740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 0x56010b4190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feba1653740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 0x56010b430030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feba1653740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b424370 0x56010b43b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feba1653740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb981a8410 con 0x56010b424370 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feba1653740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb981aa010 con 0x56010b41f2e0 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feba1653740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb981a8ae0 con 0x56010b41e9a0 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb977fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 0x56010b4190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b424370 0x56010b43b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb97fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 0x56010b430030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b424370 0x56010b43b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37508/0 (socket says 172.21.15.103:37508) 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb977fe640 1 -- 172.21.15.103:0/740586022 learned_addr learned my addr 172.21.15.103:0/740586022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb97fff640 1 -- 172.21.15.103:0/740586022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 msgr2=0x56010b4190b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb97fff640 1 --2- 172.21.15.103:0/740586022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 0x56010b4190b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb97fff640 1 -- 172.21.15.103:0/740586022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b424370 msgr2=0x56010b43b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb97fff640 1 --2- 172.21.15.103:0/740586022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b424370 0x56010b43b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb97fff640 1 -- 172.21.15.103:0/740586022 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010b4315b0 con 0x56010b41f2e0 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.226+0000 7feb977fe640 1 --2- 172.21.15.103:0/740586022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b424370 0x56010b43b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.227+0000 7feb97fff640 1 --2- 172.21.15.103:0/740586022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 0x56010b430030 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x56010b437100 tx=0x7feb880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493a313e72068947 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.227+0000 7feb7dffb640 1 -- 172.21.15.103:0/740586022 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88003d80 con 0x56010b41f2e0 2024-09-05T02:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.227+0000 7feb7dffb640 1 -- 172.21.15.103:0/740586022 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb88012900 con 0x56010b41f2e0 2024-09-05T02:08:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.227+0000 7feb7dffb640 1 -- 172.21.15.103:0/740586022 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8801e880 con 0x56010b41f2e0 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 -- 172.21.15.103:0/740586022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 msgr2=0x56010b430030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 --2- 172.21.15.103:0/740586022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 0x56010b430030 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x56010b437100 tx=0x7feb880046c0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 -- 172.21.15.103:0/740586022 shutdown_connections 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 --2- 172.21.15.103:0/740586022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b424370 0x56010b43b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 --2- 172.21.15.103:0/740586022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b41f2e0 0x56010b430030 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 --2- 172.21.15.103:0/740586022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 0x56010b4190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 -- 172.21.15.103:0/740586022 >> 172.21.15.103:0/740586022 conn(0x56010b41a010 msgr2=0x56010b4189a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 -- 172.21.15.103:0/740586022 shutdown_connections 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 -- 172.21.15.103:0/740586022 wait complete. 2024-09-05T02:08:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.228+0000 7feba1653740 1 -- start start 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feba1653740 1 Processor -- start 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feba1653740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 0x56010b42bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feba1653740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41f2e0 0x56010b42c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feba1653740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 0x56010b42f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feba1653740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb981a9940 con 0x56010b41f2e0 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feba1653740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb981a91b0 con 0x56010b424370 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feba1653740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb981a63f0 con 0x56010b41e9a0 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feba1653740 10 client.0 _ll_register_callbacks cb 0x7feb880095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb97fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41f2e0 0x56010b42c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb97fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41f2e0 0x56010b42c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37512/0 (socket says 172.21.15.103:37512) 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb97fff640 1 -- 172.21.15.103:0/3606345754 learned_addr learned my addr 172.21.15.103:0/3606345754 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb977fe640 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 0x56010b42bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb977fe640 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 0x56010b42f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb977fe640 1 -- 172.21.15.103:0/3606345754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 msgr2=0x56010b42f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb977fe640 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 0x56010b42f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb977fe640 1 -- 172.21.15.103:0/3606345754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41f2e0 msgr2=0x56010b42c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb977fe640 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41f2e0 0x56010b42c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.229+0000 7feb977fe640 1 -- 172.21.15.103:0/3606345754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56010b42f8c0 con 0x56010b41e9a0 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feb977fe640 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 0x56010b42bd50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feb8c029bd0 tx=0x7feb8c009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a413deef98f9ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feb777fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feb777fe640 1 -- 172.21.15.103:0/3606345754 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c041020 con 0x56010b41e9a0 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feb777fe640 1 -- 172.21.15.103:0/3606345754 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feb8c029630 con 0x56010b41e9a0 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feba1653740 1 -- 172.21.15.103:0/3606345754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb88012900 con 0x56010b41e9a0 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feb777fe640 1 -- 172.21.15.103:0/3606345754 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7feb8c02e070 con 0x56010b41e9a0 2024-09-05T02:08:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feba1653740 10 client.25483 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feba1653740 1 -- 172.21.15.103:0/3606345754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56010b4a3b50 con 0x56010b41e9a0 2024-09-05T02:08:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feb777fe640 1 -- 172.21.15.103:0/3606345754 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb88012900 con 0x56010b41e9a0 2024-09-05T02:08:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feb777fe640 1 -- 172.21.15.103:0/3606345754 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7feb8c027910 con 0x56010b41e9a0 2024-09-05T02:08:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feba1653740 10 client.25483 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feba1653740 10 client.25483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.230+0000 7feba1653740 20 client.25483 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.231+0000 7feba1653740 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56010b434580 0x56010b4a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.231+0000 7feba1653740 4 client.25483 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:08:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.231+0000 7feba1653740 1 -- 172.21.15.103:0/3606345754 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56010b389d70 con 0x56010b434580 2024-09-05T02:08:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.231+0000 7feb97fff640 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56010b434580 0x56010b4a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.231+0000 7feb777fe640 10 client.25483 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.231+0000 7feb97fff640 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56010b434580 0x56010b4a63f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee0c36770f7da56 server_cookie=71ddc773672d7eee in_seq=0 out_seq=0 2024-09-05T02:08:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feb777fe640 1 -- 172.21.15.103:0/3606345754 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56010b389d70 con 0x56010b434580 2024-09-05T02:08:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feb777fe640 10 client.25483 handle_command_reply: tid=0 2024-09-05T02:08:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 client.25483 shutdown 2024-09-05T02:08:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 -- 172.21.15.103:0/3606345754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 msgr2=0x56010b42bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 0x56010b42bd50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feb8c029bd0 tx=0x7feb8c009850 comp rx=0 tx=0).stop 2024-09-05T02:08:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 -- 172.21.15.103:0/3606345754 shutdown_connections 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56010b434580 0x56010b4a63f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56010b424370 0x56010b42f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feb777fe640 0 client.25483 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56010b41f2e0 0x56010b42c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 --2- 172.21.15.103:0/3606345754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56010b41e9a0 0x56010b42bd50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 -- 172.21.15.103:0/3606345754 >> 172.21.15.103:0/3606345754 conn(0x56010b41a010 msgr2=0x56010b4189a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 -- 172.21.15.103:0/3606345754 shutdown_connections 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 1 -- 172.21.15.103:0/3606345754 wait complete. 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.245+0000 7feba1653740 20 client.25483 trim_cache size 0 max 16384 2024-09-05T02:08:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:08:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:08:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:08:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:08:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 -- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feb800a2c30 msgr2=0x7feb800a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feb800a2c30 0x7feb800a50f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feb900b9290 tx=0x7feb90087b20 comp rx=0 tx=0).stop 2024-09-05T02:08:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 -- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 msgr2=0x7feb9819e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 0x7feb9819e160 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feb90031970 tx=0x7feb90003ba0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 -- 172.21.15.103:0/2224312152 shutdown_connections 2024-09-05T02:08:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb98102610 0x7feb9819e160 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb98101c50 0x7feb981970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feb800a2c30 0x7feb800a50f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 --2- 172.21.15.103:0/2224312152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb98100a60 0x7feb98196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 -- 172.21.15.103:0/2224312152 >> 172.21.15.103:0/2224312152 conn(0x7feb980fc1e0 msgr2=0x7feb98105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 -- 172.21.15.103:0/2224312152 shutdown_connections 2024-09-05T02:08:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:28.246+0000 7feb9e8b7640 1 -- 172.21.15.103:0/2224312152 wait complete. 2024-09-05T02:08:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:28 smithi167 ceph-mon[35122]: pgmap v531: 129 pgs: 129 active+clean; 354 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 553 KiB/s wr, 83 op/s 2024-09-05T02:08:28.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:28 smithi139 ceph-mon[34262]: pgmap v531: 129 pgs: 129 active+clean; 354 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 553 KiB/s wr, 83 op/s 2024-09-05T02:08:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:28 smithi103 ceph-mon[30902]: pgmap v531: 129 pgs: 129 active+clean; 354 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 553 KiB/s wr, 83 op/s 2024-09-05T02:08:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:30 smithi167 ceph-mon[35122]: pgmap v532: 129 pgs: 129 active+clean; 354 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 510 KiB/s wr, 79 op/s 2024-09-05T02:08:30.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:30 smithi139 ceph-mon[34262]: pgmap v532: 129 pgs: 129 active+clean; 354 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 510 KiB/s wr, 79 op/s 2024-09-05T02:08:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:30 smithi103 ceph-mon[30902]: pgmap v532: 129 pgs: 129 active+clean; 354 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 510 KiB/s wr, 79 op/s 2024-09-05T02:08:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:31 smithi167 ceph-mon[35122]: pgmap v533: 129 pgs: 129 active+clean; 354 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 391 KiB/s wr, 42 op/s 2024-09-05T02:08:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:31 smithi139 ceph-mon[34262]: pgmap v533: 129 pgs: 129 active+clean; 354 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 391 KiB/s wr, 42 op/s 2024-09-05T02:08:31.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:31 smithi103 ceph-mon[30902]: pgmap v533: 129 pgs: 129 active+clean; 354 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 391 KiB/s wr, 42 op/s 2024-09-05T02:08:33.875 INFO:tasks.workunit.client.0.smithi103.stdout: 59 7458 0 4853 0 2023 0 2024-09-05T02:08:34.629 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:34 smithi103 ceph-mon[30902]: pgmap v534: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 466 KiB/s wr, 31 op/s 2024-09-05T02:08:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:34 smithi167 ceph-mon[35122]: pgmap v534: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 466 KiB/s wr, 31 op/s 2024-09-05T02:08:34.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:34 smithi139 ceph-mon[34262]: pgmap v534: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 466 KiB/s wr, 31 op/s 2024-09-05T02:08:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:36 smithi167 ceph-mon[35122]: pgmap v535: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 312 KiB/s wr, 29 op/s 2024-09-05T02:08:36.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:36 smithi139 ceph-mon[34262]: pgmap v535: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 312 KiB/s wr, 29 op/s 2024-09-05T02:08:36.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:36 smithi103 ceph-mon[30902]: pgmap v535: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 312 KiB/s wr, 29 op/s 2024-09-05T02:08:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:38 smithi167 ceph-mon[35122]: pgmap v536: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 22 op/s 2024-09-05T02:08:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:38 smithi139 ceph-mon[34262]: pgmap v536: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 22 op/s 2024-09-05T02:08:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:38 smithi103 ceph-mon[30902]: pgmap v536: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 22 op/s 2024-09-05T02:08:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:40 smithi167 ceph-mon[35122]: pgmap v537: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 25 op/s 2024-09-05T02:08:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:40 smithi139 ceph-mon[34262]: pgmap v537: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 25 op/s 2024-09-05T02:08:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:40 smithi103 ceph-mon[30902]: pgmap v537: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 25 op/s 2024-09-05T02:08:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 -- 172.21.15.103:0/2771167392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 msgr2=0x7fe83c108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 --2- 172.21.15.103:0/2771167392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 0x7fe83c108ea0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe834009ca0 tx=0x7fe83402a8c0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 -- 172.21.15.103:0/2771167392 shutdown_connections 2024-09-05T02:08:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 --2- 172.21.15.103:0/2771167392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 0x7fe83c108ea0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 --2- 172.21.15.103:0/2771167392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c101c50 0x7fe83c1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 --2- 172.21.15.103:0/2771167392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe83c100a60 0x7fe83c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 -- 172.21.15.103:0/2771167392 >> 172.21.15.103:0/2771167392 conn(0x7fe83c0fc1e0 msgr2=0x7fe83c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 -- 172.21.15.103:0/2771167392 shutdown_connections 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 -- 172.21.15.103:0/2771167392 wait complete. 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 Processor -- start 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.346+0000 7fe84383c640 1 -- start start 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe84383c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c100a60 0x7fe83c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe84383c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe83c101c50 0x7fe83c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe84383c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 0x7fe83c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe84383c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c10c5f0 con 0x7fe83c102610 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe84383c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c10c470 con 0x7fe83c101c50 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe84383c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c10c770 con 0x7fe83c100a60 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe841dd9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 0x7fe83c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe841dd9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 0x7fe83c19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55440/0 (socket says 172.21.15.103:55440) 2024-09-05T02:08:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe841dd9640 1 -- 172.21.15.103:0/2193152202 learned_addr learned my addr 172.21.15.103:0/2193152202 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe8415d8640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c100a60 0x7fe83c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe841dd9640 1 -- 172.21.15.103:0/2193152202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c100a60 msgr2=0x7fe83c197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe841dd9640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c100a60 0x7fe83c197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe841dd9640 1 -- 172.21.15.103:0/2193152202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe83c101c50 msgr2=0x7fe83c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe840dd7640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe83c101c50 0x7fe83c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe841dd9640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe83c101c50 0x7fe83c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe841dd9640 1 -- 172.21.15.103:0/2193152202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c19f940 con 0x7fe83c102610 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe840dd7640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe83c101c50 0x7fe83c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.347+0000 7fe841dd9640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 0x7fe83c19f2a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe834009dc0 tx=0x7fe834003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.348+0000 7fe8327fc640 1 -- 172.21.15.103:0/2193152202 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834004040 con 0x7fe83c102610 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.348+0000 7fe8327fc640 1 -- 172.21.15.103:0/2193152202 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe834004690 con 0x7fe83c102610 2024-09-05T02:08:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.348+0000 7fe84383c640 1 -- 172.21.15.103:0/2193152202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c19fbd0 con 0x7fe83c102610 2024-09-05T02:08:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.348+0000 7fe8327fc640 1 -- 172.21.15.103:0/2193152202 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83402f4c0 con 0x7fe83c102610 2024-09-05T02:08:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.348+0000 7fe84383c640 1 -- 172.21.15.103:0/2193152202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c1a0120 con 0x7fe83c102610 2024-09-05T02:08:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.350+0000 7fe8327fc640 1 -- 172.21.15.103:0/2193152202 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe834030650 con 0x7fe83c102610 2024-09-05T02:08:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.350+0000 7fe8465d8740 1 Processor -- start 2024-09-05T02:08:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.350+0000 7fe8465d8740 1 -- start start 2024-09-05T02:08:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.350+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 0x5653fccbfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc02e0 0x5653fccc4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc5370 0x5653fccc7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8465d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5653fcc232f0 con 0x5653fccc02e0 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8465d8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5653fcbcb270 con 0x5653fccc5370 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8465d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5653fcc1f2b0 con 0x5653fccbf9a0 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8415d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc02e0 0x5653fccc4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8415d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc02e0 0x5653fccc4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55452/0 (socket says 172.21.15.103:55452) 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8415d8640 1 -- 172.21.15.103:0/3209806214 learned_addr learned my addr 172.21.15.103:0/3209806214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe840dd7640 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 0x5653fccbfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8327fc640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe8280a2c30 0x7fe8280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe840dd7640 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc5370 0x5653fccc7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8415d8640 1 -- 172.21.15.103:0/3209806214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 msgr2=0x5653fccbfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8415d8640 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 0x5653fccbfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8415d8640 1 -- 172.21.15.103:0/3209806214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc5370 msgr2=0x5653fccc7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8415d8640 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc5370 0x5653fccc7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8415d8640 1 -- 172.21.15.103:0/3209806214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653fcbca340 con 0x5653fccc02e0 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe841dd9640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe8280a2c30 0x7fe8280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe840dd7640 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc5370 0x5653fccc7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8327fc640 1 -- 172.21.15.103:0/2193152202 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe8340f0d70 con 0x7fe83c102610 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8415d8640 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc02e0 0x5653fccc4da0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe83800cba0 tx=0x7fe83800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a6b0f496be9cea server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.351+0000 7fe8277fe640 1 -- 172.21.15.103:0/3209806214 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838010630 con 0x5653fccc02e0 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8277fe640 1 -- 172.21.15.103:0/3209806214 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe838010c80 con 0x5653fccc02e0 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8277fe640 1 -- 172.21.15.103:0/3209806214 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83801ab60 con 0x5653fccc02e0 2024-09-05T02:08:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe841dd9640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe8280a2c30 0x7fe8280a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe8340ee830 tx=0x7fe8340e7db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8465d8740 1 -- 172.21.15.103:0/3209806214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc02e0 msgr2=0x5653fccc4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8465d8740 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc02e0 0x5653fccc4da0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe83800cba0 tx=0x7fe83800e720 comp rx=0 tx=0).stop 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8465d8740 1 -- 172.21.15.103:0/3209806214 shutdown_connections 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8465d8740 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc5370 0x5653fccc7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8465d8740 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc02e0 0x5653fccc4da0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8465d8740 1 --2- 172.21.15.103:0/3209806214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 0x5653fccbfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8465d8740 1 -- 172.21.15.103:0/3209806214 >> 172.21.15.103:0/3209806214 conn(0x5653fccbb010 msgr2=0x5653fccbb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8465d8740 1 -- 172.21.15.103:0/3209806214 shutdown_connections 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.352+0000 7fe8465d8740 1 -- 172.21.15.103:0/3209806214 wait complete. 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8465d8740 1 -- start start 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8465d8740 1 Processor -- start 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 0x5653fccc9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fccca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccd1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8465d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5653fccc8340 con 0x5653fccc5370 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8465d8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5653fcbac290 con 0x5653fccc02e0 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8465d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5653fccc84c0 con 0x5653fccbf9a0 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8465d8740 10 client.0 _ll_register_callbacks cb 0x5653fcbdd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe840dd7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccd1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe840dd7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccd1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55462/0 (socket says 172.21.15.103:55462) 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe840dd7640 1 -- 172.21.15.103:0/2732295542 learned_addr learned my addr 172.21.15.103:0/2732295542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe840dd7640 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 0x5653fccc9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.353+0000 7fe8415d8640 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fccca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe840dd7640 1 -- 172.21.15.103:0/2732295542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 msgr2=0x5653fccc9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe840dd7640 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 0x5653fccc9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe840dd7640 1 -- 172.21.15.103:0/2732295542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 msgr2=0x5653fccca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe840dd7640 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fccca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe840dd7640 1 -- 172.21.15.103:0/2732295542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653fc936210 con 0x5653fccc5370 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe8415d8640 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fccca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe840dd7640 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccd1170 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe82c017e00 tx=0x7fe82c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2426da1254d5782 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe824ff9640 1 -- 172.21.15.103:0/2732295542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c00f040 con 0x5653fccc5370 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe8465d8740 1 -- 172.21.15.103:0/2732295542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe838010c80 con 0x5653fccc5370 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe824ff9640 1 -- 172.21.15.103:0/2732295542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe82c0155e0 con 0x5653fccc5370 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe824ff9640 1 -- 172.21.15.103:0/2732295542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe82c028050 con 0x5653fccc5370 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe8465d8740 10 client.16134 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe8465d8740 1 -- 172.21.15.103:0/2732295542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5653fca02620 con 0x5653fccc5370 2024-09-05T02:08:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.354+0000 7fe824ff9640 1 -- 172.21.15.103:0/2732295542 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe838010c80 con 0x5653fccc5370 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.355+0000 7fe824ff9640 1 -- 172.21.15.103:0/2732295542 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fe82c023440 con 0x5653fccc5370 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.355+0000 7fe8465d8740 10 client.16134 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.355+0000 7fe8465d8740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.355+0000 7fe8465d8740 20 client.16134 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.355+0000 7fe8465d8740 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5653fccd5820 0x5653fccd7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.355+0000 7fe8465d8740 4 client.16134 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.355+0000 7fe8465d8740 1 -- 172.21.15.103:0/2732295542 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653fcc30a20 con 0x5653fccd5820 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.355+0000 7fe8415d8640 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5653fccd5820 0x5653fccd7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe824ff9640 10 client.16134 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8415d8640 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5653fccd5820 0x5653fccd7c70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967d2d208e4912cd server_cookie=b265156fcb33468e in_seq=0 out_seq=0 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe824ff9640 1 -- 172.21.15.103:0/2732295542 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653fcc30a20 con 0x5653fccd5820 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe824ff9640 10 client.16134 handle_command_reply: tid=0 2024-09-05T02:08:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8465d8740 1 client.16134 shutdown 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8465d8740 1 -- 172.21.15.103:0/2732295542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 msgr2=0x5653fccd1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8465d8740 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccd1170 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe82c017e00 tx=0x7fe82c0047d0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8465d8740 1 -- 172.21.15.103:0/2732295542 shutdown_connections 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8465d8740 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5653fccd5820 0x5653fccd7c70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8465d8740 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccd1170 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8465d8740 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fccca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe824ff9640 0 client.16134 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8465d8740 1 --2- 172.21.15.103:0/2732295542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccbf9a0 0x5653fccc9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.356+0000 7fe8465d8740 1 -- 172.21.15.103:0/2732295542 >> 172.21.15.103:0/2732295542 conn(0x5653fccbb010 msgr2=0x5653fccbc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.357+0000 7fe8465d8740 1 -- 172.21.15.103:0/2732295542 shutdown_connections 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.357+0000 7fe8465d8740 1 -- 172.21.15.103:0/2732295542 wait complete. 2024-09-05T02:08:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.357+0000 7fe8465d8740 20 client.16134 trim_cache size 0 max 16384 2024-09-05T02:08:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe8465d8740 1 Processor -- start 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe8465d8740 1 -- start start 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccbf9a0 0x5653fccbbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccc02e0 0x5653fccd1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccbc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe8465d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1aa6a0 con 0x5653fccc5370 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe8465d8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1ac2a0 con 0x5653fccbf9a0 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe8465d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1aad70 con 0x5653fccc02e0 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe840dd7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccbc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe840dd7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccbc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55472/0 (socket says 172.21.15.103:55472) 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe840dd7640 1 -- 172.21.15.103:0/3937953085 learned_addr learned my addr 172.21.15.103:0/3937953085 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.371+0000 7fe840dd7640 1 --2- 172.21.15.103:0/3937953085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccbf9a0 0x5653fccbbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe8415d8640 1 --2- 172.21.15.103:0/3937953085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccc02e0 0x5653fccd1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe840dd7640 1 -- 172.21.15.103:0/3937953085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccc02e0 msgr2=0x5653fccd1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe840dd7640 1 --2- 172.21.15.103:0/3937953085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccc02e0 0x5653fccd1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe840dd7640 1 -- 172.21.15.103:0/3937953085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccbf9a0 msgr2=0x5653fccbbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe840dd7640 1 --2- 172.21.15.103:0/3937953085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccbf9a0 0x5653fccbbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe840dd7640 1 -- 172.21.15.103:0/3937953085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653fccc4780 con 0x5653fccc5370 2024-09-05T02:08:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe840dd7640 1 --2- 172.21.15.103:0/3937953085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccbc400 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0179f0 tx=0x7fe82c01b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1188d420961a4688 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe826ffd640 1 -- 172.21.15.103:0/3937953085 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c00cc20 con 0x5653fccc5370 2024-09-05T02:08:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe826ffd640 1 -- 172.21.15.103:0/3937953085 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe82c01f070 con 0x5653fccc5370 2024-09-05T02:08:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.372+0000 7fe826ffd640 1 -- 172.21.15.103:0/3937953085 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0235f0 con 0x5653fccc5370 2024-09-05T02:08:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 -- 172.21.15.103:0/3937953085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 msgr2=0x5653fccbc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 --2- 172.21.15.103:0/3937953085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccbc400 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0179f0 tx=0x7fe82c01b040 comp rx=0 tx=0).stop 2024-09-05T02:08:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 -- 172.21.15.103:0/3937953085 shutdown_connections 2024-09-05T02:08:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 --2- 172.21.15.103:0/3937953085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccc5370 0x5653fccbc400 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 --2- 172.21.15.103:0/3937953085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccc02e0 0x5653fccd1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 --2- 172.21.15.103:0/3937953085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccbf9a0 0x5653fccbbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 -- 172.21.15.103:0/3937953085 >> 172.21.15.103:0/3937953085 conn(0x5653fccbb010 msgr2=0x5653fccc9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 -- 172.21.15.103:0/3937953085 shutdown_connections 2024-09-05T02:08:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 -- 172.21.15.103:0/3937953085 wait complete. 2024-09-05T02:08:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 -- start start 2024-09-05T02:08:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 Processor -- start 2024-09-05T02:08:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.373+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccbf9a0 0x5653fccce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fcccec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8465d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccc5370 0x5653fcccfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8465d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1abbd0 con 0x5653fccbf9a0 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8465d8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1ab440 con 0x5653fccc02e0 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8465d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1a8680 con 0x5653fccc5370 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8465d8740 10 client.0 _ll_register_callbacks cb 0x7fe82c00ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe840dd7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccbf9a0 0x5653fccce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe840dd7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccbf9a0 0x5653fccce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55482/0 (socket says 172.21.15.103:55482) 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe840dd7640 1 -- 172.21.15.103:0/528861319 learned_addr learned my addr 172.21.15.103:0/528861319 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe840dd7640 1 -- 172.21.15.103:0/528861319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccc5370 msgr2=0x5653fcccfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe840dd7640 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccc5370 0x5653fcccfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe840dd7640 1 -- 172.21.15.103:0/528861319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 msgr2=0x5653fcccec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8415d8640 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fcccec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe840dd7640 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fcccec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe840dd7640 1 -- 172.21.15.103:0/528861319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653fccd02a0 con 0x5653fccbf9a0 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8415d8640 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fcccec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe840dd7640 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccbf9a0 0x5653fccce740 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe82c004810 tx=0x7fe82c017f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9026b28cf890232a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe824ff9640 1 -- 172.21.15.103:0/528861319 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c00cb30 con 0x5653fccbf9a0 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe824ff9640 1 -- 172.21.15.103:0/528861319 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe82c0233b0 con 0x5653fccbf9a0 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8465d8740 1 -- 172.21.15.103:0/528861319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe82c01f070 con 0x5653fccbf9a0 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe824ff9640 1 -- 172.21.15.103:0/528861319 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe82c0235d0 con 0x5653fccbf9a0 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8465d8740 10 client.16140 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.374+0000 7fe8465d8740 1 -- 172.21.15.103:0/528861319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5653fcd40550 con 0x5653fccbf9a0 2024-09-05T02:08:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe824ff9640 1 -- 172.21.15.103:0/528861319 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe82c01f070 con 0x5653fccbf9a0 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe824ff9640 1 -- 172.21.15.103:0/528861319 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fe82c02c8f0 con 0x5653fccbf9a0 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe8465d8740 10 client.16140 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe8465d8740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe8465d8740 20 client.16140 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe8465d8740 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5653fcd40e70 0x5653fcd43290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe8465d8740 4 client.16140 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe8465d8740 1 -- 172.21.15.103:0/528861319 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653fcc2ad70 con 0x5653fcd40e70 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe8415d8640 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5653fcd40e70 0x5653fcd43290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe824ff9640 10 client.16140 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.375+0000 7fe8415d8640 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5653fcd40e70 0x5653fcd43290 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e1478ac8bc3927 server_cookie=bed0ecd1119679fb in_seq=0 out_seq=0 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe824ff9640 1 -- 172.21.15.103:0/528861319 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653fcc2ad70 con 0x5653fcd40e70 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe824ff9640 10 client.16140 handle_command_reply: tid=0 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 client.16140 shutdown 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 -- 172.21.15.103:0/528861319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccbf9a0 msgr2=0x5653fccce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccbf9a0 0x5653fccce740 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe82c004810 tx=0x7fe82c017f80 comp rx=0 tx=0).stop 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 -- 172.21.15.103:0/528861319 shutdown_connections 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x5653fcd40e70 0x5653fcd43290 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5653fccc5370 0x5653fcccfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653fccc02e0 0x5653fcccec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe824ff9640 0 client.16140 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 --2- 172.21.15.103:0/528861319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5653fccbf9a0 0x5653fccce740 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 -- 172.21.15.103:0/528861319 >> 172.21.15.103:0/528861319 conn(0x5653fccbb010 msgr2=0x5653fccb99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 -- 172.21.15.103:0/528861319 shutdown_connections 2024-09-05T02:08:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 1 -- 172.21.15.103:0/528861319 wait complete. 2024-09-05T02:08:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.381+0000 7fe8465d8740 20 client.16140 trim_cache size 0 max 16384 2024-09-05T02:08:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:08:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:08:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:08:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:08:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 -- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe8280a2c30 msgr2=0x7fe8280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe8280a2c30 0x7fe8280a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe8340ee830 tx=0x7fe8340e7db0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 -- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 msgr2=0x7fe83c19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 0x7fe83c19f2a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe834009dc0 tx=0x7fe834003ba0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 -- 172.21.15.103:0/2193152202 shutdown_connections 2024-09-05T02:08:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe83c102610 0x7fe83c19f2a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe83c101c50 0x7fe83c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe8280a2c30 0x7fe8280a50f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 --2- 172.21.15.103:0/2193152202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c100a60 0x7fe83c197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 -- 172.21.15.103:0/2193152202 >> 172.21.15.103:0/2193152202 conn(0x7fe83c0fc1e0 msgr2=0x7fe83c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 -- 172.21.15.103:0/2193152202 shutdown_connections 2024-09-05T02:08:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:41.382+0000 7fe84383c640 1 -- 172.21.15.103:0/2193152202 wait complete. 2024-09-05T02:08:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:42 smithi167 ceph-mon[35122]: pgmap v538: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 16 op/s 2024-09-05T02:08:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:42 smithi139 ceph-mon[34262]: pgmap v538: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 16 op/s 2024-09-05T02:08:42.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:42 smithi103 ceph-mon[30902]: pgmap v538: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 16 op/s 2024-09-05T02:08:43.721 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:08:43.876 INFO:tasks.workunit.client.0.smithi103.stdout: 59 0 0 0 0 0 0 2024-09-05T02:08:43.981 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:08:44.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:44 smithi103 ceph-mon[30902]: pgmap v539: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 13 op/s 2024-09-05T02:08:44.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 -- 172.21.15.103:0/3038193106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8103700 msgr2=0x7feaf810bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:44.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 --2- 172.21.15.103:0/3038193106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8103700 0x7feaf810bbf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7feaec009900 tx=0x7feaec02a480 comp rx=0 tx=0).stop 2024-09-05T02:08:44.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 -- 172.21.15.103:0/3038193106 shutdown_connections 2024-09-05T02:08:44.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 --2- 172.21.15.103:0/3038193106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaf810c180 0x7feaf810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 --2- 172.21.15.103:0/3038193106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8103700 0x7feaf810bbf0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 --2- 172.21.15.103:0/3038193106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf8102de0 0x7feaf81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 -- 172.21.15.103:0/3038193106 >> 172.21.15.103:0/3038193106 conn(0x7feaf80fc7a0 msgr2=0x7feaf80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:44.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 -- 172.21.15.103:0/3038193106 shutdown_connections 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 -- 172.21.15.103:0/3038193106 wait complete. 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.480+0000 7feafd547640 1 Processor -- start 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feafd547640 1 -- start start 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feafd547640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf8102de0 0x7feaf8100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feafd547640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaf8103700 0x7feaf8100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feafd547640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf810c180 0x7feaf81014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feafd547640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8111e20 con 0x7feaf810c180 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feafd547640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8111ca0 con 0x7feaf8103700 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feafd547640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8111fa0 con 0x7feaf8102de0 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf810c180 0x7feaf81014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf810c180 0x7feaf81014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55502/0 (socket says 172.21.15.103:55502) 2024-09-05T02:08:44.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf77fe640 1 -- 172.21.15.103:0/3565195631 learned_addr learned my addr 172.21.15.103:0/3565195631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf67fc640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaf8103700 0x7feaf8100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf6ffd640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf8102de0 0x7feaf8100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf77fe640 1 -- 172.21.15.103:0/3565195631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf8102de0 msgr2=0x7feaf8100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf77fe640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf8102de0 0x7feaf8100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf77fe640 1 -- 172.21.15.103:0/3565195631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaf8103700 msgr2=0x7feaf8100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf77fe640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaf8103700 0x7feaf8100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf77fe640 1 -- 172.21.15.103:0/3565195631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf81a6800 con 0x7feaf810c180 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf6ffd640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf8102de0 0x7feaf8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.481+0000 7feaf77fe640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf810c180 0x7feaf81014c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7feaf000cce0 tx=0x7feaf000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.482+0000 7feadffff640 1 -- 172.21.15.103:0/3565195631 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf000f450 con 0x7feaf810c180 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.482+0000 7feadffff640 1 -- 172.21.15.103:0/3565195631 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf000faa0 con 0x7feaf810c180 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.482+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf81a6af0 con 0x7feaf810c180 2024-09-05T02:08:44.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.482+0000 7feadffff640 1 -- 172.21.15.103:0/3565195631 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0018b70 con 0x7feaf810c180 2024-09-05T02:08:44.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.482+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf81a6fb0 con 0x7feaf810c180 2024-09-05T02:08:44.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.483+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf81031c0 con 0x7feaf810c180 2024-09-05T02:08:44.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.484+0000 7feadffff640 1 -- 172.21.15.103:0/3565195631 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feaf0018d10 con 0x7feaf810c180 2024-09-05T02:08:44.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.484+0000 7feadffff640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feae00a2b80 0x7feae00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:44.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.485+0000 7feaf6ffd640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feae00a2b80 0x7feae00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:44.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.485+0000 7feadffff640 1 -- 172.21.15.103:0/3565195631 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feaf00cf7c0 con 0x7feaf810c180 2024-09-05T02:08:44.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.485+0000 7feaf6ffd640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feae00a2b80 0x7feae00a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7feae80058c0 tx=0x7feae8005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:44.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.489+0000 7feadffff640 1 -- 172.21.15.103:0/3565195631 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feaf0014030 con 0x7feaf810c180 2024-09-05T02:08:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:44 smithi167 ceph-mon[35122]: pgmap v539: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 13 op/s 2024-09-05T02:08:44.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:44 smithi139 ceph-mon[34262]: pgmap v539: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 13 op/s 2024-09-05T02:08:44.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.691+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaf8107200 con 0x7feaf810c180 2024-09-05T02:08:44.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.692+0000 7feadffff640 1 -- 172.21.15.103:0/3565195631 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+12005 (secure 0 0 0) 0x7feaf009c370 con 0x7feaf810c180 2024-09-05T02:08:44.694 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:08:44.694 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":35,"btime":"2024-09-05T02:06:39:264756+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:06:38.694889+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24457,24464,24466,24497]},"id":1}]} 2024-09-05T02:08:44.694 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 35 2024-09-05T02:08:44.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feae00a2b80 msgr2=0x7feae00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:44.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feae00a2b80 0x7feae00a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7feae80058c0 tx=0x7feae8005ec0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf810c180 msgr2=0x7feaf81014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:44.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf810c180 0x7feaf81014c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7feaf000cce0 tx=0x7feaf000a620 comp rx=0 tx=0).stop 2024-09-05T02:08:44.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 shutdown_connections 2024-09-05T02:08:44.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf810c180 0x7feaf81014c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaf8103700 0x7feaf8100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7feae00a2b80 0x7feae00a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 --2- 172.21.15.103:0/3565195631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf8102de0 0x7feaf8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:44.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 >> 172.21.15.103:0/3565195631 conn(0x7feaf80fc7a0 msgr2=0x7feaf80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:44.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 shutdown_connections 2024-09-05T02:08:44.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:44.694+0000 7feafd547640 1 -- 172.21.15.103:0/3565195631 wait complete. 2024-09-05T02:08:45.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:45 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3565195631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:08:45.527 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 8e1959f2 --release --await' 2024-09-05T02:08:45.528 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 8e1959f2 --release --await 2024-09-05T02:08:45.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:45 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3565195631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:08:45.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:45 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3565195631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:08:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:08:46.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.288+0000 7f6f97f47640 1 -- 172.21.15.103:0/204477275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90100e20 msgr2=0x7f6f90101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:46.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.288+0000 7f6f97f47640 1 --2- 172.21.15.103:0/204477275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90100e20 0x7f6f90101200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009900 tx=0x7f6f8802a480 comp rx=0 tx=0).stop 2024-09-05T02:08:46.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 -- 172.21.15.103:0/204477275 shutdown_connections 2024-09-05T02:08:46.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 --2- 172.21.15.103:0/204477275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f90102ef0 0x7f6f9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 --2- 172.21.15.103:0/204477275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90102570 0x7f6f901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 --2- 172.21.15.103:0/204477275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90100e20 0x7f6f90101200 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 -- 172.21.15.103:0/204477275 >> 172.21.15.103:0/204477275 conn(0x7f6f900fc7d0 msgr2=0x7f6f900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:46.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 -- 172.21.15.103:0/204477275 shutdown_connections 2024-09-05T02:08:46.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 -- 172.21.15.103:0/204477275 wait complete. 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 Processor -- start 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 -- start start 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90100e20 0x7f6f90199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90102570 0x7f6f90199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f90102ef0 0x7f6f901a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90110e50 con 0x7f6f90102570 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90110cd0 con 0x7f6f90102ef0 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.289+0000 7f6f97f47640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90110fd0 con 0x7f6f90100e20 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f954e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90102570 0x7f6f90199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f954e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90102570 0x7f6f90199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55522/0 (socket says 172.21.15.103:55522) 2024-09-05T02:08:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f954e2640 1 -- 172.21.15.103:0/1030600612 learned_addr learned my addr 172.21.15.103:0/1030600612 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f964e4640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f90102ef0 0x7f6f901a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90100e20 0x7f6f90199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f954e2640 1 -- 172.21.15.103:0/1030600612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90100e20 msgr2=0x7f6f90199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f954e2640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90100e20 0x7f6f90199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f954e2640 1 -- 172.21.15.103:0/1030600612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f90102ef0 msgr2=0x7f6f901a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f954e2640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f90102ef0 0x7f6f901a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f954e2640 1 -- 172.21.15.103:0/1030600612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f901a48e0 con 0x7f6f90102570 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90100e20 0x7f6f90199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f954e2640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90102570 0x7f6f90199860 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400ba50 tx=0x7f6f8400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f8effd640 1 -- 172.21.15.103:0/1030600612 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84019070 con 0x7f6f90102570 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f8effd640 1 -- 172.21.15.103:0/1030600612 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f8400cbe0 con 0x7f6f90102570 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f97f47640 1 -- 172.21.15.103:0/1030600612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f901a4ab0 con 0x7f6f90102570 2024-09-05T02:08:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f8effd640 1 -- 172.21.15.103:0/1030600612 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840075d0 con 0x7f6f90102570 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.290+0000 7f6f97f47640 1 -- 172.21.15.103:0/1030600612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f901a4fc0 con 0x7f6f90102570 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f8effd640 1 -- 172.21.15.103:0/1030600612 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6f8401c070 con 0x7f6f90102570 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f9a320740 1 Processor -- start 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f9a320740 1 -- start start 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 0x55e19a82d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 0x55e19a837d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a838240 0x55e19a83a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f9a320740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e19a792620 con 0x55e19a838240 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f9a320740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e19a739e50 con 0x55e19a82d060 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f9a320740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e19a78e5e0 con 0x55e19a82d980 2024-09-05T02:08:46.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f954e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a838240 0x55e19a83a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f964e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 0x55e19a82d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f954e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a838240 0x55e19a83a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55526/0 (socket says 172.21.15.103:55526) 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f95ce3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 0x55e19a837d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.293+0000 7f6f954e2640 1 -- 172.21.15.103:0/3050517508 learned_addr learned my addr 172.21.15.103:0/3050517508 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f8effd640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f780a2b80 0x7f6f780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f954e2640 1 -- 172.21.15.103:0/3050517508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 msgr2=0x55e19a837d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f954e2640 1 --2- 172.21.15.103:0/3050517508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 0x55e19a837d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f954e2640 1 -- 172.21.15.103:0/3050517508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 msgr2=0x55e19a82d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f954e2640 1 --2- 172.21.15.103:0/3050517508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 0x55e19a82d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f954e2640 1 -- 172.21.15.103:0/3050517508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e19a72ad40 con 0x55e19a838240 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f964e4640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f780a2b80 0x7f6f780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f954e2640 1 --2- 172.21.15.103:0/3050517508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a838240 0x55e19a83a630 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6f840c80c0 tx=0x7f6f840c59f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86f97a3cadd39b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f8effd640 1 -- 172.21.15.103:0/1030600612 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6f840ce9b0 con 0x7f6f90102570 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f77fff640 1 -- 172.21.15.103:0/3050517508 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840cee60 con 0x55e19a838240 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f77fff640 1 -- 172.21.15.103:0/3050517508 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f840c9310 con 0x55e19a838240 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.294+0000 7f6f77fff640 1 -- 172.21.15.103:0/3050517508 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8409a870 con 0x55e19a838240 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 -- 172.21.15.103:0/3050517508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a838240 msgr2=0x55e19a83a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3050517508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a838240 0x55e19a83a630 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6f840c80c0 tx=0x7f6f840c59f0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f964e4640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f780a2b80 0x7f6f780a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000b7a0 tx=0x7f6f800046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 -- 172.21.15.103:0/3050517508 shutdown_connections 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3050517508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a838240 0x55e19a83a630 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3050517508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 0x55e19a837d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3050517508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 0x55e19a82d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 -- 172.21.15.103:0/3050517508 >> 172.21.15.103:0/3050517508 conn(0x55e19a8295f0 msgr2=0x55e19a8299d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 -- 172.21.15.103:0/3050517508 shutdown_connections 2024-09-05T02:08:46.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 -- 172.21.15.103:0/3050517508 wait complete. 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 -- start start 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.295+0000 7f6f9a320740 1 Processor -- start 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a8350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d980 0x55e19a835630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 0x55e19a835b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f9a320740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e19a83ae10 con 0x55e19a82d060 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f9a320740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e19a71af10 con 0x55e19a82d980 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f9a320740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e19a83af90 con 0x55e19a838240 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f9a320740 10 client.0 _ll_register_callbacks cb 0x55e19a73edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f95ce3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a8350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f95ce3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a8350f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55530/0 (socket says 172.21.15.103:55530) 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f95ce3640 1 -- 172.21.15.103:0/2538908306 learned_addr learned my addr 172.21.15.103:0/2538908306 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f964e4640 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 0x55e19a835b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f954e2640 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d980 0x55e19a835630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f95ce3640 1 -- 172.21.15.103:0/2538908306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 msgr2=0x55e19a835b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 0x55e19a835b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f95ce3640 1 -- 172.21.15.103:0/2538908306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d980 msgr2=0x55e19a835630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d980 0x55e19a835630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f95ce3640 1 -- 172.21.15.103:0/2538908306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e19a4a1580 con 0x55e19a82d060 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f954e2640 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d980 0x55e19a835630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.296+0000 7f6f964e4640 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 0x55e19a835b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a8350f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009900 tx=0x7f6f88031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e64a85acb3f9c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f757fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f757fa640 1 -- 172.21.15.103:0/2538908306 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88035040 con 0x55e19a82d060 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f757fa640 1 -- 172.21.15.103:0/2538908306 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6f88004250 con 0x55e19a82d060 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f9a320740 1 -- 172.21.15.103:0/2538908306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e19a79fb90 con 0x55e19a82d060 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f757fa640 1 -- 172.21.15.103:0/2538908306 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6f88004450 con 0x55e19a82d060 2024-09-05T02:08:46.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f9a320740 10 client.16158 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f9a320740 1 -- 172.21.15.103:0/2538908306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e19a56b960 con 0x55e19a82d060 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f757fa640 1 -- 172.21.15.103:0/2538908306 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e19a79fb90 con 0x55e19a82d060 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f757fa640 1 -- 172.21.15.103:0/2538908306 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f6f880046c0 con 0x55e19a82d060 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f9a320740 10 client.16158 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f9a320740 10 client.16158 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f9a320740 20 client.16158 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55e19a847d80 0x55e19a84a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f9a320740 4 client.16158 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.297+0000 7f6f9a320740 1 -- 172.21.15.103:0/2538908306 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e19a79a0a0 con 0x55e19a847d80 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.298+0000 7f6f954e2640 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55e19a847d80 0x55e19a84a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.298+0000 7f6f954e2640 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55e19a847d80 0x55e19a84a1e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87dfbb0cc3a2531 server_cookie=8b0534f4d99d07f7 in_seq=0 out_seq=0 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.298+0000 7f6f757fa640 10 client.16158 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f757fa640 1 -- 172.21.15.103:0/2538908306 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e19a79a0a0 con 0x55e19a847d80 2024-09-05T02:08:46.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f757fa640 10 client.16158 handle_command_reply: tid=0 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 client.16158 shutdown 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 -- 172.21.15.103:0/2538908306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 msgr2=0x55e19a8350f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a8350f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009900 tx=0x7f6f88031430 comp rx=0 tx=0).stop 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 -- 172.21.15.103:0/2538908306 shutdown_connections 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55e19a847d80 0x55e19a84a1e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 0x55e19a835b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d980 0x55e19a835630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f757fa640 0 client.16158 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2538908306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a8350f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 -- 172.21.15.103:0/2538908306 >> 172.21.15.103:0/2538908306 conn(0x55e19a8295f0 msgr2=0x55e19a82f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 -- 172.21.15.103:0/2538908306 shutdown_connections 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 1 -- 172.21.15.103:0/2538908306 wait complete. 2024-09-05T02:08:46.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.299+0000 7f6f9a320740 20 client.16158 trim_cache size 0 max 16384 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f9a320740 1 Processor -- start 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f9a320740 1 -- start start 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 0x55e19a836fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d980 0x55e19a82fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 0x55e19a84fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f9a320740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901acb50 con 0x55e19a82d980 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f9a320740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901ae750 con 0x55e19a82d060 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f9a320740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901ad220 con 0x55e19a838240 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f954e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d980 0x55e19a82fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f954e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d980 0x55e19a82fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55540/0 (socket says 172.21.15.103:55540) 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.316+0000 7f6f954e2640 1 -- 172.21.15.103:0/2196280471 learned_addr learned my addr 172.21.15.103:0/2196280471 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/2196280471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 0x55e19a836fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f954e2640 1 -- 172.21.15.103:0/2196280471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 msgr2=0x55e19a84fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f954e2640 1 --2- 172.21.15.103:0/2196280471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 0x55e19a84fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f954e2640 1 -- 172.21.15.103:0/2196280471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 msgr2=0x55e19a836fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f954e2640 1 --2- 172.21.15.103:0/2196280471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 0x55e19a836fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f954e2640 1 -- 172.21.15.103:0/2196280471 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e19a829090 con 0x55e19a82d980 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/2196280471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 0x55e19a836fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f954e2640 1 --2- 172.21.15.103:0/2196280471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d980 0x55e19a82fac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f840cc150 tx=0x7f6f840c59f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd75ed3f1c81e8fb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f777fe640 1 -- 172.21.15.103:0/2196280471 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840cee60 con 0x55e19a82d980 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f777fe640 1 -- 172.21.15.103:0/2196280471 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f8409f210 con 0x55e19a82d980 2024-09-05T02:08:46.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.317+0000 7f6f777fe640 1 -- 172.21.15.103:0/2196280471 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8409b170 con 0x55e19a82d980 2024-09-05T02:08:46.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 -- 172.21.15.103:0/2196280471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d980 msgr2=0x55e19a82fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:46.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2196280471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d980 0x55e19a82fac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f840cc150 tx=0x7f6f840c59f0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 -- 172.21.15.103:0/2196280471 shutdown_connections 2024-09-05T02:08:46.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2196280471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a838240 0x55e19a84fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2196280471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d980 0x55e19a82fac0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 --2- 172.21.15.103:0/2196280471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a82d060 0x55e19a836fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 -- 172.21.15.103:0/2196280471 >> 172.21.15.103:0/2196280471 conn(0x55e19a8295f0 msgr2=0x55e19a828750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 -- 172.21.15.103:0/2196280471 shutdown_connections 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 -- 172.21.15.103:0/2196280471 wait complete. 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 -- start start 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.318+0000 7f6f9a320740 1 Processor -- start 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a840880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 0x55e19a83cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f9a320740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a838240 0x55e19a83d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f9a320740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901ae080 con 0x55e19a82d060 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f9a320740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901ad8f0 con 0x55e19a838240 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f9a320740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901aab30 con 0x55e19a82d980 2024-09-05T02:08:46.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f95ce3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a840880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f9a320740 10 client.0 _ll_register_callbacks cb 0x7f6f88032c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f95ce3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a840880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55542/0 (socket says 172.21.15.103:55542) 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f95ce3640 1 -- 172.21.15.103:0/3551504948 learned_addr learned my addr 172.21.15.103:0/3551504948 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f964e4640 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a838240 0x55e19a83d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f954e2640 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 0x55e19a83cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f95ce3640 1 -- 172.21.15.103:0/3551504948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 msgr2=0x55e19a83cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 0x55e19a83cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f95ce3640 1 -- 172.21.15.103:0/3551504948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a838240 msgr2=0x55e19a83d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a838240 0x55e19a83d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f95ce3640 1 -- 172.21.15.103:0/3551504948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e19a83d930 con 0x55e19a82d060 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f95ce3640 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a840880 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6f880096c0 tx=0x7f6f88031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd45c5d42a6a6063 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:46.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.319+0000 7f6f757fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:08:46.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.322+0000 7f6f757fa640 1 -- 172.21.15.103:0/3551504948 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88035070 con 0x55e19a82d060 2024-09-05T02:08:46.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.322+0000 7f6f757fa640 1 -- 172.21.15.103:0/3551504948 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6f88004fa0 con 0x55e19a82d060 2024-09-05T02:08:46.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.322+0000 7f6f9a320740 1 -- 172.21.15.103:0/3551504948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f8409f210 con 0x55e19a82d060 2024-09-05T02:08:46.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f757fa640 1 -- 172.21.15.103:0/3551504948 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6f8805e050 con 0x55e19a82d060 2024-09-05T02:08:46.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f9a320740 10 client.16164 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:46.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f9a320740 1 -- 172.21.15.103:0/3551504948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e19a8b3a00 con 0x55e19a82d060 2024-09-05T02:08:46.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f757fa640 1 -- 172.21.15.103:0/3551504948 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f8409f210 con 0x55e19a82d060 2024-09-05T02:08:46.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f757fa640 1 -- 172.21.15.103:0/3551504948 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f6f880324d0 con 0x55e19a82d060 2024-09-05T02:08:46.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f9a320740 10 client.16164 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:46.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f9a320740 10 client.16164 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:08:46.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f9a320740 20 client.16164 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:46.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55e19a8b41a0 0x55e19a8b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:46.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f9a320740 4 client.16164 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8e1959f2", "release": true, "await": true}] 2024-09-05T02:08:46.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f9a320740 1 -- 172.21.15.103:0/3551504948 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8e1959f2", "release": true, "await": true}) -- 0x55e19a7165a0 con 0x55e19a8b41a0 2024-09-05T02:08:46.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.323+0000 7f6f954e2640 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55e19a8b41a0 0x55e19a8b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:46.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.324+0000 7f6f954e2640 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55e19a8b41a0 0x55e19a8b65c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b7e8f1596075e server_cookie=cdafc18564d8a490 in_seq=0 out_seq=0 2024-09-05T02:08:46.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.324+0000 7f6f757fa640 10 client.16164 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:46.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f757fa640 1 -- 172.21.15.103:0/3551504948 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e19a7165a0 con 0x55e19a8b41a0 2024-09-05T02:08:46.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f757fa640 10 client.16164 handle_command_reply: tid=0 2024-09-05T02:08:46.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f9a320740 1 client.16164 shutdown 2024-09-05T02:08:46.499 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:46 smithi103 ceph-mon[30902]: pgmap v540: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-05T02:08:46.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f9a320740 1 -- 172.21.15.103:0/3551504948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 msgr2=0x55e19a840880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:46.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a840880 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6f880096c0 tx=0x7f6f88031b20 comp rx=0 tx=0).stop 2024-09-05T02:08:46.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f9a320740 1 -- 172.21.15.103:0/3551504948 shutdown_connections 2024-09-05T02:08:46.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55e19a8b41a0 0x55e19a8b65c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f757fa640 0 client.16164 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:46.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e19a838240 0x55e19a83d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e19a82d980 0x55e19a83cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f9a320740 1 --2- 172.21.15.103:0/3551504948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e19a82d060 0x55e19a840880 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.497+0000 7f6f9a320740 1 -- 172.21.15.103:0/3551504948 >> 172.21.15.103:0/3551504948 conn(0x55e19a8295f0 msgr2=0x55e19a843060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.498+0000 7f6f9a320740 1 -- 172.21.15.103:0/3551504948 shutdown_connections 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.498+0000 7f6f9a320740 1 -- 172.21.15.103:0/3551504948 wait complete. 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.498+0000 7f6f9a320740 20 client.16164 trim_cache size 0 max 16384 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 35, 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 20, 2024-09-05T02:08:46.501 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "8e1959f2": { 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 20, 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 154.3, 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-05T02:08:46.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-05T02:08:46.503 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-05T02:08:46.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:46.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:46.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:46.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:46.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:08:46.503 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 -- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f780a2b80 msgr2=0x7f6f780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f780a2b80 0x7f6f780a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000b7a0 tx=0x7f6f800046c0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 -- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90102570 msgr2=0x7f6f90199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90102570 0x7f6f90199860 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400ba50 tx=0x7f6f8400bf10 comp rx=0 tx=0).stop 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 -- 172.21.15.103:0/1030600612 shutdown_connections 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f90102ef0 0x7f6f901a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f90102570 0x7f6f90199860 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f6f780a2b80 0x7f6f780a5040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 --2- 172.21.15.103:0/1030600612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90100e20 0x7f6f90199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 -- 172.21.15.103:0/1030600612 >> 172.21.15.103:0/1030600612 conn(0x7f6f900fc7d0 msgr2=0x7f6f900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 -- 172.21.15.103:0/1030600612 shutdown_connections 2024-09-05T02:08:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:08:46.499+0000 7f6f97f47640 1 -- 172.21.15.103:0/1030600612 wait complete. 2024-09-05T02:08:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:46 smithi167 ceph-mon[35122]: pgmap v540: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-05T02:08:46.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:46 smithi139 ceph-mon[34262]: pgmap v540: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-05T02:08:47.364 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8e1959f2', total seconds elapsed: 38.0 2024-09-05T02:08:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:48 smithi167 ceph-mon[35122]: pgmap v541: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-09-05T02:08:48.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:48 smithi139 ceph-mon[34262]: pgmap v541: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-09-05T02:08:48.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:48 smithi103 ceph-mon[30902]: pgmap v541: 129 pgs: 129 active+clean; 356 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-09-05T02:08:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:50 smithi167 ceph-mon[35122]: pgmap v542: 129 pgs: 129 active+clean; 357 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 107 KiB/s wr, 25 op/s 2024-09-05T02:08:50.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:50 smithi139 ceph-mon[34262]: pgmap v542: 129 pgs: 129 active+clean; 357 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 107 KiB/s wr, 25 op/s 2024-09-05T02:08:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:50 smithi103 ceph-mon[30902]: pgmap v542: 129 pgs: 129 active+clean; 357 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 107 KiB/s wr, 25 op/s 2024-09-05T02:08:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:52 smithi167 ceph-mon[35122]: pgmap v543: 129 pgs: 129 active+clean; 359 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 297 KiB/s wr, 34 op/s 2024-09-05T02:08:52.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:52 smithi139 ceph-mon[34262]: pgmap v543: 129 pgs: 129 active+clean; 359 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 297 KiB/s wr, 34 op/s 2024-09-05T02:08:52.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:52 smithi103 ceph-mon[30902]: pgmap v543: 129 pgs: 129 active+clean; 359 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 297 KiB/s wr, 34 op/s 2024-09-05T02:08:53.876 INFO:tasks.workunit.client.0.smithi103.stdout: 61 8876 77 5956 71 3595 56 2024-09-05T02:08:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.511+0000 7f91d60a4640 1 -- 172.21.15.103:0/3620720003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d01025c0 msgr2=0x7f91d01049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.511+0000 7f91d60a4640 1 --2- 172.21.15.103:0/3620720003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d01025c0 0x7f91d01049b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009900 tx=0x7f91c002a410 comp rx=0 tx=0).stop 2024-09-05T02:08:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.511+0000 7f91d60a4640 1 -- 172.21.15.103:0/3620720003 shutdown_connections 2024-09-05T02:08:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.511+0000 7f91d60a4640 1 --2- 172.21.15.103:0/3620720003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91d0107820 0x7f91d0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.511+0000 7f91d60a4640 1 --2- 172.21.15.103:0/3620720003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d0104ef0 0x7f91d01072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.511+0000 7f91d60a4640 1 --2- 172.21.15.103:0/3620720003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d01025c0 0x7f91d01049b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.511+0000 7f91d60a4640 1 -- 172.21.15.103:0/3620720003 >> 172.21.15.103:0/3620720003 conn(0x7f91d00fc1e0 msgr2=0x7f91d00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.511+0000 7f91d60a4640 1 -- 172.21.15.103:0/3620720003 shutdown_connections 2024-09-05T02:08:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.512+0000 7f91d60a4640 1 -- 172.21.15.103:0/3620720003 wait complete. 2024-09-05T02:08:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91d60a4640 1 Processor -- start 2024-09-05T02:08:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91d60a4640 1 -- start start 2024-09-05T02:08:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91d60a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91d01025c0 0x7f91d0101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91d60a4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d0104ef0 0x7f91d0100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91d60a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d0107820 0x7f91d0100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91d60a4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91d010d6b0 con 0x7f91d0107820 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91d60a4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91d010d530 con 0x7f91d01025c0 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91d60a4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91d010d830 con 0x7f91d0104ef0 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d0107820 0x7f91d0100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d0104ef0 0x7f91d0100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d0104ef0 0x7f91d0100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:40386/0 (socket says 172.21.15.103:40386) 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d0107820 0x7f91d0100ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44946/0 (socket says 172.21.15.103:44946) 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91ceffd640 1 -- 172.21.15.103:0/1458099229 learned_addr learned my addr 172.21.15.103:0/1458099229 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.513+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91d01025c0 0x7f91d0101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91ceffd640 1 -- 172.21.15.103:0/1458099229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91d01025c0 msgr2=0x7f91d0101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91ceffd640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91d01025c0 0x7f91d0101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91ceffd640 1 -- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d0107820 msgr2=0x7f91d0100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91ceffd640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d0107820 0x7f91d0100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91ceffd640 1 -- 172.21.15.103:0/1458099229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d0100ff0 con 0x7f91d0104ef0 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91cffff640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d0107820 0x7f91d0100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91ceffd640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d0104ef0 0x7f91d0100570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e9c0 tx=0x7f91c400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91ccff9640 1 -- 172.21.15.103:0/1458099229 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c400ccc0 con 0x7f91d0104ef0 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91d60a4640 1 -- 172.21.15.103:0/1458099229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d01012e0 con 0x7f91d0104ef0 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91ccff9640 1 -- 172.21.15.103:0/1458099229 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91c40054d0 con 0x7f91d0104ef0 2024-09-05T02:08:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91ccff9640 1 -- 172.21.15.103:0/1458099229 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4010430 con 0x7f91d0104ef0 2024-09-05T02:08:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.514+0000 7f91d60a4640 1 -- 172.21.15.103:0/1458099229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d01a6870 con 0x7f91d0104ef0 2024-09-05T02:08:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.517+0000 7f91ccff9640 1 -- 172.21.15.103:0/1458099229 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f91c4005020 con 0x7f91d0104ef0 2024-09-05T02:08:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.517+0000 7f91d8e37740 1 Processor -- start 2024-09-05T02:08:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91d8e37740 1 -- start start 2024-09-05T02:08:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91ccff9640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91b80a2cb0 0x7f91b80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dc1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dc6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 0x55b767dc9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91d8e37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b767d252f0 con 0x55b767dc19a0 2024-09-05T02:08:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91d8e37740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b767ccd270 con 0x55b767dc7370 2024-09-05T02:08:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91d8e37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b767d212b0 con 0x55b767dc22e0 2024-09-05T02:08:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91ccff9640 1 -- 172.21.15.103:0/1458099229 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f91c40cfa00 con 0x7f91d0104ef0 2024-09-05T02:08:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dc1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91cffff640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91b80a2cb0 0x7f91b80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dc1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44956/0 (socket says 172.21.15.103:44956) 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91cf7fe640 1 -- 172.21.15.103:0/3406663547 learned_addr learned my addr 172.21.15.103:0/3406663547 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.518+0000 7f91ceffd640 1 --2- 172.21.15.103:0/3406663547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dc6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91ceffd640 1 -- 172.21.15.103:0/3406663547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 msgr2=0x55b767dc9830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91ceffd640 1 --2- 172.21.15.103:0/3406663547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 0x55b767dc9830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91ceffd640 1 -- 172.21.15.103:0/3406663547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 msgr2=0x55b767dc1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91ceffd640 1 --2- 172.21.15.103:0/3406663547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dc1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91ceffd640 1 -- 172.21.15.103:0/3406663547 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b767ccc340 con 0x55b767dc22e0 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/3406663547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dc1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91cffff640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91b80a2cb0 0x7f91b80a5170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f91c80099a0 tx=0x7f91c8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91ceffd640 1 --2- 172.21.15.103:0/3406663547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dc6da0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f91c40cd4c0 tx=0x7f91c400efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7e81f1c51070b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91b5ffb640 1 -- 172.21.15.103:0/3406663547 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c409b810 con 0x55b767dc22e0 2024-09-05T02:08:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91b5ffb640 1 -- 172.21.15.103:0/3406663547 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91c4005310 con 0x55b767dc22e0 2024-09-05T02:08:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.519+0000 7f91b5ffb640 1 -- 172.21.15.103:0/3406663547 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c409a240 con 0x55b767dc22e0 2024-09-05T02:08:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.520+0000 7f91d8e37740 1 -- 172.21.15.103:0/3406663547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 msgr2=0x55b767dc6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.520+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3406663547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dc6da0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f91c40cd4c0 tx=0x7f91c400efd0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.520+0000 7f91d8e37740 1 -- 172.21.15.103:0/3406663547 shutdown_connections 2024-09-05T02:08:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.520+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3406663547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 0x55b767dc9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.520+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3406663547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dc6da0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.520+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3406663547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dc1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.520+0000 7f91d8e37740 1 -- 172.21.15.103:0/3406663547 >> 172.21.15.103:0/3406663547 conn(0x55b767dbd010 msgr2=0x55b767dbd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.520+0000 7f91d8e37740 1 -- 172.21.15.103:0/3406663547 shutdown_connections 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.520+0000 7f91d8e37740 1 -- 172.21.15.103:0/3406663547 wait complete. 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91d8e37740 1 -- start start 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91d8e37740 1 Processor -- start 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dcbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dcc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 0x55b767dd31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91d8e37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b767dca340 con 0x55b767dc19a0 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91d8e37740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b767cae290 con 0x55b767dc7370 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91d8e37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b767dca4c0 con 0x55b767dc22e0 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91d8e37740 10 client.0 _ll_register_callbacks cb 0x55b767cdf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dcbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.521+0000 7f91cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 0x55b767dd31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dcc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dcc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:40412/0 (socket says 172.21.15.103:40412) 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91ceffd640 1 -- 172.21.15.103:0/2042953959 learned_addr learned my addr 172.21.15.103:0/2042953959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 0x55b767dd31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:49340/0 (socket says 172.21.15.103:49340) 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dcbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44964/0 (socket says 172.21.15.103:44964) 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91ceffd640 1 -- 172.21.15.103:0/2042953959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 msgr2=0x55b767dd31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91ceffd640 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 0x55b767dd31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91ceffd640 1 -- 172.21.15.103:0/2042953959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 msgr2=0x55b767dcbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91ceffd640 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dcbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91ceffd640 1 -- 172.21.15.103:0/2042953959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b767a38210 con 0x55b767dc22e0 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dcbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91cffff640 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 0x55b767dd31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:08:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91ceffd640 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dcc190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91c409bbb0 tx=0x7f91c409b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f375d553ac3d3080 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-09-05T02:08:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91aeffd640 1 -- 172.21.15.103:0/2042953959 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40a0260 con 0x55b767dc22e0 2024-09-05T02:08:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91aeffd640 1 -- 172.21.15.103:0/2042953959 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f91c40607c0 con 0x55b767dc22e0 2024-09-05T02:08:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.522+0000 7f91d8e37740 1 -- 172.21.15.103:0/2042953959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b767d32a20 con 0x55b767dc22e0 2024-09-05T02:08:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91aeffd640 1 -- 172.21.15.103:0/2042953959 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f91c409c5b0 con 0x55b767dc22e0 2024-09-05T02:08:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91d8e37740 10 client.25516 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91d8e37740 1 -- 172.21.15.103:0/2042953959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b767b04620 con 0x55b767dc22e0 2024-09-05T02:08:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91aeffd640 1 -- 172.21.15.103:0/2042953959 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91c40609c0 con 0x55b767dc22e0 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91aeffd640 1 -- 172.21.15.103:0/2042953959 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f91c409af70 con 0x55b767dc22e0 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91d8e37740 10 client.25516 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91d8e37740 10 client.25516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91d8e37740 20 client.25516 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91d8e37740 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b767dd7980 0x55b767dd9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91d8e37740 4 client.25516 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91d8e37740 1 -- 172.21.15.103:0/2042953959 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b767d32a20 con 0x55b767dd7980 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.523+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b767dd7980 0x55b767dd9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.524+0000 7f91aeffd640 10 client.25516 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.524+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b767dd7980 0x55b767dd9d70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba16eb5a0511e216 server_cookie=157094964e944ea6 in_seq=0 out_seq=0 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.524+0000 7f91aeffd640 1 -- 172.21.15.103:0/2042953959 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b767d32a20 con 0x55b767dd7980 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.524+0000 7f91aeffd640 10 client.25516 handle_command_reply: tid=0 2024-09-05T02:08:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 client.25516 shutdown 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 -- 172.21.15.103:0/2042953959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 msgr2=0x55b767dcc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dcc190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91c409bbb0 tx=0x7f91c409b920 comp rx=0 tx=0).stop 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 -- 172.21.15.103:0/2042953959 shutdown_connections 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b767dd7980 0x55b767dd9d70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91aeffd640 0 client.25516 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc7370 0x55b767dd31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dcc190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 --2- 172.21.15.103:0/2042953959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dcbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 -- 172.21.15.103:0/2042953959 >> 172.21.15.103:0/2042953959 conn(0x55b767dbd010 msgr2=0x55b767dbe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 -- 172.21.15.103:0/2042953959 shutdown_connections 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 1 -- 172.21.15.103:0/2042953959 wait complete. 2024-09-05T02:08:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.525+0000 7f91d8e37740 20 client.25516 trim_cache size 0 max 16384 2024-09-05T02:08:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.545+0000 7f91d8e37740 1 Processor -- start 2024-09-05T02:08:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.545+0000 7f91d8e37740 1 -- start start 2024-09-05T02:08:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc19a0 0x55b767dc1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dd3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc7370 0x55b767dbdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91d8e37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b0d10 con 0x55b767dc7370 2024-09-05T02:08:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91d8e37740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b2910 con 0x55b767dc19a0 2024-09-05T02:08:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91d8e37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b13e0 con 0x55b767dc22e0 2024-09-05T02:08:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc7370 0x55b767dbdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc19a0 0x55b767dc1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc7370 0x55b767dbdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44978/0 (socket says 172.21.15.103:44978) 2024-09-05T02:08:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dd3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91cffff640 1 -- 172.21.15.103:0/3954245809 learned_addr learned my addr 172.21.15.103:0/3954245809 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91cf7fe640 1 -- 172.21.15.103:0/3954245809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 msgr2=0x55b767dd3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/3954245809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dd3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91cf7fe640 1 -- 172.21.15.103:0/3954245809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc7370 msgr2=0x55b767dbdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/3954245809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc7370 0x55b767dbdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.546+0000 7f91cf7fe640 1 -- 172.21.15.103:0/3954245809 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b767dc6800 con 0x55b767dc19a0 2024-09-05T02:08:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.547+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/3954245809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc19a0 0x55b767dc1d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91c0005c40 tx=0x7f91c0005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b48e810b1d4258e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.547+0000 7f91b57fa640 1 -- 172.21.15.103:0/3954245809 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0035040 con 0x55b767dc19a0 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.547+0000 7f91b57fa640 1 -- 172.21.15.103:0/3954245809 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91c004a070 con 0x55b767dc19a0 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.547+0000 7f91b57fa640 1 -- 172.21.15.103:0/3954245809 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0004b50 con 0x55b767dc19a0 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 -- 172.21.15.103:0/3954245809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc19a0 msgr2=0x55b767dc1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3954245809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc19a0 0x55b767dc1d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91c0005c40 tx=0x7f91c0005a20 comp rx=0 tx=0).stop 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 -- 172.21.15.103:0/3954245809 shutdown_connections 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3954245809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc7370 0x55b767dbdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3954245809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc22e0 0x55b767dd3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3954245809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc19a0 0x55b767dc1d80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 -- 172.21.15.103:0/3954245809 >> 172.21.15.103:0/3954245809 conn(0x55b767dbd010 msgr2=0x55b767dcbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 -- 172.21.15.103:0/3954245809 shutdown_connections 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 -- 172.21.15.103:0/3954245809 wait complete. 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.548+0000 7f91d8e37740 1 -- start start 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91d8e37740 1 Processor -- start 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dd2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc22e0 0x55b767dcfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91d8e37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc7370 0x55b767dcff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91d8e37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b2240 con 0x55b767dc19a0 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91d8e37740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01b1ab0 con 0x55b767dc22e0 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91d8e37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01aecf0 con 0x55b767dc7370 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91d8e37740 10 client.0 _ll_register_callbacks cb 0x7f91c80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:08:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dd2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc22e0 0x55b767dcfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91cffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc7370 0x55b767dcff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dd2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44984/0 (socket says 172.21.15.103:44984) 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91cffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc7370 0x55b767dcff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:40432/0 (socket says 172.21.15.103:40432) 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91cf7fe640 1 -- 172.21.15.103:0/3227586842 learned_addr learned my addr 172.21.15.103:0/3227586842 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc22e0 0x55b767dcfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:49354/0 (socket says 172.21.15.103:49354) 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91ceffd640 1 -- 172.21.15.103:0/3227586842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc7370 msgr2=0x55b767dcff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91ceffd640 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc7370 0x55b767dcff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91ceffd640 1 -- 172.21.15.103:0/3227586842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 msgr2=0x55b767dd2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91ceffd640 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dd2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.549+0000 7f91ceffd640 1 -- 172.21.15.103:0/3227586842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b767dd05e0 con 0x55b767dc22e0 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91ceffd640 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc22e0 0x55b767dcfa00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f91c400cfd0 tx=0x7f91c409a1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68dee4385f3895a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:08:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91aeffd640 1 -- 172.21.15.103:0/3227586842 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40bacf0 con 0x55b767dc22e0 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91aeffd640 1 -- 172.21.15.103:0/3227586842 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f91c4097c00 con 0x55b767dc22e0 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91d8e37740 1 -- 172.21.15.103:0/3227586842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c004a070 con 0x55b767dc22e0 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91aeffd640 1 -- 172.21.15.103:0/3227586842 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f91c40a0260 con 0x55b767dc22e0 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91d8e37740 10 client.25577 fetch_fsmap learned FSMap version 35 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91d8e37740 1 -- 172.21.15.103:0/3227586842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b767e41720 con 0x55b767dc22e0 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91aeffd640 1 -- 172.21.15.103:0/3227586842 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91c004a070 con 0x55b767dc22e0 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.550+0000 7f91aeffd640 1 -- 172.21.15.103:0/3227586842 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f91c4097e00 con 0x55b767dc22e0 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.551+0000 7f91d8e37740 10 client.25577 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.551+0000 7f91d8e37740 10 client.25577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.551+0000 7f91d8e37740 20 client.25577 populate_metadata read hostname 'smithi103' 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.551+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b767dd7980 0x55b767e43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:08:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.551+0000 7f91d8e37740 4 client.25577 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:08:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.551+0000 7f91d8e37740 1 -- 172.21.15.103:0/3227586842 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b767d2cd70 con 0x55b767dd7980 2024-09-05T02:08:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.551+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b767dd7980 0x55b767e43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:08:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.551+0000 7f91aeffd640 10 client.25577 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.551+0000 7f91cf7fe640 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b767dd7980 0x55b767e43fa0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f69d7b35e0c5f1 server_cookie=b8679b52d2712956 in_seq=0 out_seq=0 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91aeffd640 1 -- 172.21.15.103:0/3227586842 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b767d2cd70 con 0x55b767dd7980 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91aeffd640 10 client.25577 handle_command_reply: tid=0 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 client.25577 shutdown 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 -- 172.21.15.103:0/3227586842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc22e0 msgr2=0x55b767dcfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc22e0 0x55b767dcfa00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f91c400cfd0 tx=0x7f91c409a1b0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 -- 172.21.15.103:0/3227586842 shutdown_connections 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55b767dd7980 0x55b767e43fa0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91aeffd640 0 client.25577 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b767dc7370 0x55b767dcff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b767dc22e0 0x55b767dcfa00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 --2- 172.21.15.103:0/3227586842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b767dc19a0 0x55b767dd2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 -- 172.21.15.103:0/3227586842 >> 172.21.15.103:0/3227586842 conn(0x55b767dbd010 msgr2=0x55b767dbb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 -- 172.21.15.103:0/3227586842 shutdown_connections 2024-09-05T02:08:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.606+0000 7f91d8e37740 1 -- 172.21.15.103:0/3227586842 wait complete. 2024-09-05T02:08:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.607+0000 7f91d8e37740 20 client.25577 trim_cache size 0 max 16384 2024-09-05T02:08:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:08:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:08:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:08:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:08:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.607+0000 7f91d60a4640 1 -- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91b80a2cb0 msgr2=0x7f91b80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.607+0000 7f91d60a4640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91b80a2cb0 0x7f91b80a5170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f91c80099a0 tx=0x7f91c8009500 comp rx=0 tx=0).stop 2024-09-05T02:08:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.607+0000 7f91d60a4640 1 -- 172.21.15.103:0/1458099229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d0104ef0 msgr2=0x7f91d0100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:08:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.607+0000 7f91d60a4640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d0104ef0 0x7f91d0100570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e9c0 tx=0x7f91c400edc0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.607+0000 7f91d60a4640 1 -- 172.21.15.103:0/1458099229 shutdown_connections 2024-09-05T02:08:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.607+0000 7f91d60a4640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d0107820 0x7f91d0100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.608+0000 7f91d60a4640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f91b80a2cb0 0x7f91b80a5170 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.608+0000 7f91d60a4640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d0104ef0 0x7f91d0100570 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.608+0000 7f91d60a4640 1 --2- 172.21.15.103:0/1458099229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91d01025c0 0x7f91d0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:08:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.608+0000 7f91d60a4640 1 -- 172.21.15.103:0/1458099229 >> 172.21.15.103:0/1458099229 conn(0x7f91d00fc1e0 msgr2=0x7f91d0102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:08:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.608+0000 7f91d60a4640 1 -- 172.21.15.103:0/1458099229 shutdown_connections 2024-09-05T02:08:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:08:54.608+0000 7f91d60a4640 1 -- 172.21.15.103:0/1458099229 wait complete. 2024-09-05T02:08:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:54 smithi167 ceph-mon[35122]: pgmap v544: 129 pgs: 129 active+clean; 361 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 514 KiB/s wr, 48 op/s 2024-09-05T02:08:54.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:54 smithi139 ceph-mon[34262]: pgmap v544: 129 pgs: 129 active+clean; 361 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 514 KiB/s wr, 48 op/s 2024-09-05T02:08:54.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:54 smithi103 ceph-mon[30902]: pgmap v544: 129 pgs: 129 active+clean; 361 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 514 KiB/s wr, 48 op/s 2024-09-05T02:08:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:56 smithi167 ceph-mon[35122]: pgmap v545: 129 pgs: 129 active+clean; 366 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 976 KiB/s wr, 100 op/s 2024-09-05T02:08:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:56 smithi139 ceph-mon[34262]: pgmap v545: 129 pgs: 129 active+clean; 366 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 976 KiB/s wr, 100 op/s 2024-09-05T02:08:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:56 smithi103 ceph-mon[30902]: pgmap v545: 129 pgs: 129 active+clean; 366 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 976 KiB/s wr, 100 op/s 2024-09-05T02:08:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:08:58 smithi167 ceph-mon[35122]: pgmap v546: 129 pgs: 129 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1008 KiB/s wr, 102 op/s 2024-09-05T02:08:58.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:08:58 smithi139 ceph-mon[34262]: pgmap v546: 129 pgs: 129 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1008 KiB/s wr, 102 op/s 2024-09-05T02:08:58.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:08:58 smithi103 ceph-mon[30902]: pgmap v546: 129 pgs: 129 active+clean; 367 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1008 KiB/s wr, 102 op/s 2024-09-05T02:09:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:00 smithi167 ceph-mon[35122]: pgmap v547: 129 pgs: 129 active+clean; 366 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T02:09:00.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:00 smithi139 ceph-mon[34262]: pgmap v547: 129 pgs: 129 active+clean; 366 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T02:09:00.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:00 smithi103 ceph-mon[30902]: pgmap v547: 129 pgs: 129 active+clean; 366 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T02:09:02.638 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:02 smithi167 ceph-mon[35122]: pgmap v548: 129 pgs: 129 active+clean; 371 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-05T02:09:02.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:02 smithi139 ceph-mon[34262]: pgmap v548: 129 pgs: 129 active+clean; 371 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-05T02:09:02.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:02 smithi103 ceph-mon[30902]: pgmap v548: 129 pgs: 129 active+clean; 371 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-05T02:09:03.876 INFO:tasks.workunit.client.0.smithi103.stdout: 61 12752 60 7947 120 3579 100 2024-09-05T02:09:04.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:04 smithi167 ceph-mon[35122]: pgmap v549: 129 pgs: 129 active+clean; 374 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-05T02:09:04.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:04 smithi139 ceph-mon[34262]: pgmap v549: 129 pgs: 129 active+clean; 374 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-05T02:09:04.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:04 smithi103 ceph-mon[30902]: pgmap v549: 129 pgs: 129 active+clean; 374 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-05T02:09:06.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:06 smithi167 ceph-mon[35122]: pgmap v550: 129 pgs: 129 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T02:09:06.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:06 smithi139 ceph-mon[34262]: pgmap v550: 129 pgs: 129 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T02:09:06.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:06 smithi103 ceph-mon[30902]: pgmap v550: 129 pgs: 129 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T02:09:07.550 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-09-05T02:09:07.551 INFO:tasks.workunit.client.0.smithi103.stdout:Final score for writes: 61 2024-09-05T02:09:07.551 INFO:tasks.workunit.client.0.smithi103.stdout:Final score for reads : 6941 2024-09-05T02:09:07.551 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-09-05T02:09:07.551 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-05T02:09:07.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 -- 172.21.15.103:0/2072682611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 msgr2=0x7eff64109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 --2- 172.21.15.103:0/2072682611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 0x7eff64109fa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009ca0 tx=0x7eff5c02a8c0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 -- 172.21.15.103:0/2072682611 shutdown_connections 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 --2- 172.21.15.103:0/2072682611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 0x7eff64109fa0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 --2- 172.21.15.103:0/2072682611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff64102d50 0x7eff641031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 --2- 172.21.15.103:0/2072682611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64101b60 0x7eff64101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 -- 172.21.15.103:0/2072682611 >> 172.21.15.103:0/2072682611 conn(0x7eff640fd2e0 msgr2=0x7eff640ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 -- 172.21.15.103:0/2072682611 shutdown_connections 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 -- 172.21.15.103:0/2072682611 wait complete. 2024-09-05T02:09:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 Processor -- start 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 -- start start 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64101b60 0x7eff64197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff64102d50 0x7eff64198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 0x7eff6419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff6410d690 con 0x7eff64103710 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff6410d510 con 0x7eff64102d50 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.706+0000 7eff6c5be640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff6410d810 con 0x7eff64101b60 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6a35a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64101b60 0x7eff64197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6ab5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 0x7eff6419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff69b59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff64102d50 0x7eff64198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6a35a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64101b60 0x7eff64197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:60190/0 (socket says 172.21.15.103:60190) 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6a35a640 1 -- 172.21.15.103:0/1069679557 learned_addr learned my addr 172.21.15.103:0/1069679557 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6ab5b640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 0x7eff6419f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33330/0 (socket says 172.21.15.103:33330) 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6a35a640 1 -- 172.21.15.103:0/1069679557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff64102d50 msgr2=0x7eff64198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6a35a640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff64102d50 0x7eff64198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6a35a640 1 -- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 msgr2=0x7eff6419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6a35a640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 0x7eff6419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6a35a640 1 -- 172.21.15.103:0/1069679557 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6419f910 con 0x7eff64101b60 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6ab5b640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 0x7eff6419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6a35a640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64101b60 0x7eff64197d00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9c0 tx=0x7eff6000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff5b7fe640 1 -- 172.21.15.103:0/1069679557 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6000cd80 con 0x7eff64101b60 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff5b7fe640 1 -- 172.21.15.103:0/1069679557 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff600035f0 con 0x7eff64101b60 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6c5be640 1 -- 172.21.15.103:0/1069679557 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6419fc00 con 0x7eff64101b60 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff5b7fe640 1 -- 172.21.15.103:0/1069679557 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60010690 con 0x7eff64101b60 2024-09-05T02:09:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.707+0000 7eff6c5be640 1 -- 172.21.15.103:0/1069679557 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff641a0100 con 0x7eff64101b60 2024-09-05T02:09:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.710+0000 7eff5b7fe640 1 -- 172.21.15.103:0/1069679557 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7eff600037a0 con 0x7eff64101b60 2024-09-05T02:09:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.710+0000 7eff6f430740 1 Processor -- start 2024-09-05T02:09:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.710+0000 7eff6f430740 1 -- start start 2024-09-05T02:09:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.710+0000 7eff6f430740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d49a0 0x55afa83d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.710+0000 7eff6f430740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d52e0 0x55afa83d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.710+0000 7eff6f430740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83da370 0x55afa83dc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.710+0000 7eff6f430740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55afa83382f0 con 0x55afa83d49a0 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.710+0000 7eff6f430740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55afa82e0270 con 0x55afa83da370 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.710+0000 7eff6f430740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55afa83342b0 con 0x55afa83d52e0 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff6ab5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d49a0 0x55afa83d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff69b59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83da370 0x55afa83dc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff6a35a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d52e0 0x55afa83d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff69b59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83da370 0x55afa83dc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:49784/0 (socket says 172.21.15.103:49784) 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff6ab5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d49a0 0x55afa83d4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33342/0 (socket says 172.21.15.103:33342) 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff69b59640 1 -- 172.21.15.103:0/1384945657 learned_addr learned my addr 172.21.15.103:0/1384945657 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff5b7fe640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7eff500a2c30 0x7eff500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff6ab5b640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7eff500a2c30 0x7eff500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff69b59640 1 -- 172.21.15.103:0/1384945657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d52e0 msgr2=0x55afa83d9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff69b59640 1 --2- 172.21.15.103:0/1384945657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d52e0 0x55afa83d9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff69b59640 1 -- 172.21.15.103:0/1384945657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d49a0 msgr2=0x55afa83d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff69b59640 1 --2- 172.21.15.103:0/1384945657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d49a0 0x55afa83d4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff69b59640 1 -- 172.21.15.103:0/1384945657 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa82df340 con 0x55afa83da370 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff5b7fe640 1 -- 172.21.15.103:0/1069679557 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7eff600cf9b0 con 0x7eff64101b60 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff69b59640 1 --2- 172.21.15.103:0/1384945657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83da370 0x55afa83dc830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff54009cf0 tx=0x7eff54011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6852d9fc8db8d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff58bfe640 1 -- 172.21.15.103:0/1384945657 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54004230 con 0x55afa83da370 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff6ab5b640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7eff500a2c30 0x7eff500a50f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009430 tx=0x7eff5c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff58bfe640 1 -- 172.21.15.103:0/1384945657 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff540043d0 con 0x55afa83da370 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.711+0000 7eff58bfe640 1 -- 172.21.15.103:0/1384945657 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54019850 con 0x55afa83da370 2024-09-05T02:09:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 -- 172.21.15.103:0/1384945657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83da370 msgr2=0x55afa83dc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 --2- 172.21.15.103:0/1384945657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83da370 0x55afa83dc830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff54009cf0 tx=0x7eff54011810 comp rx=0 tx=0).stop 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 -- 172.21.15.103:0/1384945657 shutdown_connections 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 --2- 172.21.15.103:0/1384945657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83da370 0x55afa83dc830 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 --2- 172.21.15.103:0/1384945657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d52e0 0x55afa83d9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 --2- 172.21.15.103:0/1384945657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d49a0 0x55afa83d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 -- 172.21.15.103:0/1384945657 >> 172.21.15.103:0/1384945657 conn(0x55afa83d0010 msgr2=0x55afa83d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 -- 172.21.15.103:0/1384945657 shutdown_connections 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 -- 172.21.15.103:0/1384945657 wait complete. 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.712+0000 7eff6f430740 1 -- start start 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6f430740 1 Processor -- start 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6f430740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83deb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6f430740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83df080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6f430740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6f430740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55afa83dd340 con 0x55afa83da370 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6f430740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55afa82c1290 con 0x55afa83d52e0 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6f430740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55afa83dd4c0 con 0x55afa83d49a0 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6f430740 10 client.0 _ll_register_callbacks cb 0x55afa82f2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff69b59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6ab5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6a35a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83deb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff69b59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83df080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:49786/0 (socket says 172.21.15.103:49786) 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff69b59640 1 -- 172.21.15.103:0/146970416 learned_addr learned my addr 172.21.15.103:0/146970416 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6ab5b640 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83e60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33350/0 (socket says 172.21.15.103:33350) 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff69b59640 1 -- 172.21.15.103:0/146970416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 msgr2=0x55afa83deb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff69b59640 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83deb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff69b59640 1 -- 172.21.15.103:0/146970416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 msgr2=0x55afa83e60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff69b59640 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83e60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff69b59640 1 -- 172.21.15.103:0/146970416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa804b210 con 0x55afa83d52e0 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6a35a640 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff6ab5b640 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff69b59640 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83df080 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7eff54004630 tx=0x7eff54003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5141a523147bff10 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.713+0000 7eff4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff4dffb640 1 -- 172.21.15.103:0/146970416 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54027070 con 0x55afa83d52e0 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6f430740 1 -- 172.21.15.103:0/146970416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff540043d0 con 0x55afa83d52e0 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff4dffb640 1 -- 172.21.15.103:0/146970416 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7eff54005240 con 0x55afa83d52e0 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff4dffb640 1 -- 172.21.15.103:0/146970416 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff54033050 con 0x55afa83d52e0 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6f430740 10 client.25589 fetch_fsmap learned FSMap version 35 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6f430740 1 -- 172.21.15.103:0/146970416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55afa8117620 con 0x55afa83d52e0 2024-09-05T02:09:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff4dffb640 1 -- 172.21.15.103:0/146970416 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff540043d0 con 0x55afa83d52e0 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff4dffb640 1 -- 172.21.15.103:0/146970416 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7eff54004c00 con 0x55afa83d52e0 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6f430740 10 client.25589 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6f430740 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6f430740 20 client.25589 populate_metadata read hostname 'smithi103' 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6f430740 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55afa83ea640 0x55afa83eca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6f430740 4 client.25589 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6f430740 1 -- 172.21.15.103:0/146970416 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afa8345a20 con 0x55afa83ea640 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.714+0000 7eff6a35a640 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55afa83ea640 0x55afa83eca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.715+0000 7eff6a35a640 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55afa83ea640 0x55afa83eca30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7334a11abba92139 server_cookie=aaf53080e2b4c4d7 in_seq=0 out_seq=0 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.715+0000 7eff4dffb640 10 client.25589 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.715+0000 7eff4dffb640 1 -- 172.21.15.103:0/146970416 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55afa8345a20 con 0x55afa83ea640 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.715+0000 7eff4dffb640 10 client.25589 handle_command_reply: tid=0 2024-09-05T02:09:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.715+0000 7eff6f430740 1 client.25589 shutdown 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 -- 172.21.15.103:0/146970416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 msgr2=0x55afa83df080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83df080 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7eff54004630 tx=0x7eff54003ec0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 -- 172.21.15.103:0/146970416 shutdown_connections 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55afa83ea640 0x55afa83eca30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff4dffb640 0 client.25589 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83df080 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 --2- 172.21.15.103:0/146970416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 -- 172.21.15.103:0/146970416 >> 172.21.15.103:0/146970416 conn(0x55afa83d0010 msgr2=0x55afa83d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 -- 172.21.15.103:0/146970416 shutdown_connections 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 1 -- 172.21.15.103:0/146970416 wait complete. 2024-09-05T02:09:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.716+0000 7eff6f430740 20 client.25589 trim_cache size 0 max 16384 2024-09-05T02:09:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6f430740 1 Processor -- start 2024-09-05T02:09:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6f430740 1 -- start start 2024-09-05T02:09:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6f430740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6f430740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83e6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6f430740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6f430740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff641aa630 con 0x55afa83da370 2024-09-05T02:09:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6f430740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff641ac230 con 0x55afa83d52e0 2024-09-05T02:09:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6f430740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff641aad00 con 0x55afa83d49a0 2024-09-05T02:09:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6ab5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6a35a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6ab5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33360/0 (socket says 172.21.15.103:33360) 2024-09-05T02:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6ab5b640 1 -- 172.21.15.103:0/1625197281 learned_addr learned my addr 172.21.15.103:0/1625197281 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff6a35a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:60218/0 (socket says 172.21.15.103:60218) 2024-09-05T02:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.731+0000 7eff69b59640 1 --2- 172.21.15.103:0/1625197281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff6a35a640 1 -- 172.21.15.103:0/1625197281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 msgr2=0x55afa83e6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff6a35a640 1 --2- 172.21.15.103:0/1625197281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83e6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff6a35a640 1 -- 172.21.15.103:0/1625197281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 msgr2=0x55afa83d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff6a35a640 1 --2- 172.21.15.103:0/1625197281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff6a35a640 1 -- 172.21.15.103:0/1625197281 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa83dcba0 con 0x55afa83d49a0 2024-09-05T02:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff6ab5b640 1 --2- 172.21.15.103:0/1625197281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff6a35a640 1 --2- 172.21.15.103:0/1625197281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83cf0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff600cd150 tx=0x7eff60099e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e7ae62a734f0b2c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff4ffff640 1 -- 172.21.15.103:0/1625197281 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600cfe60 con 0x55afa83d49a0 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff4ffff640 1 -- 172.21.15.103:0/1625197281 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff60003b10 con 0x55afa83d49a0 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.732+0000 7eff4ffff640 1 -- 172.21.15.103:0/1625197281 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600a4ad0 con 0x55afa83d49a0 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 -- 172.21.15.103:0/1625197281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 msgr2=0x55afa83cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 --2- 172.21.15.103:0/1625197281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83cf0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff600cd150 tx=0x7eff60099e70 comp rx=0 tx=0).stop 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 -- 172.21.15.103:0/1625197281 shutdown_connections 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 --2- 172.21.15.103:0/1625197281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83da370 0x55afa83d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 --2- 172.21.15.103:0/1625197281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d52e0 0x55afa83e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 --2- 172.21.15.103:0/1625197281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83d49a0 0x55afa83cf0b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 -- 172.21.15.103:0/1625197281 >> 172.21.15.103:0/1625197281 conn(0x55afa83d0010 msgr2=0x55afa83ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 -- 172.21.15.103:0/1625197281 shutdown_connections 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 -- 172.21.15.103:0/1625197281 wait complete. 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 -- start start 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 Processor -- start 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d49a0 0x55afa83e2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d52e0 0x55afa83e33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83da370 0x55afa8453300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff641abb60 con 0x55afa83d52e0 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff641ab3d0 con 0x55afa83d49a0 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff641a8610 con 0x55afa83da370 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff6f430740 10 client.0 _ll_register_callbacks cb 0x7eff60005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.733+0000 7eff69b59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d52e0 0x55afa83e33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff69b59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d52e0 0x55afa83e33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33372/0 (socket says 172.21.15.103:33372) 2024-09-05T02:09:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff69b59640 1 -- 172.21.15.103:0/3117148697 learned_addr learned my addr 172.21.15.103:0/3117148697 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6a35a640 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d49a0 0x55afa83e2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6ab5b640 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83da370 0x55afa8453300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6a35a640 1 -- 172.21.15.103:0/3117148697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83da370 msgr2=0x55afa8453300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6a35a640 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83da370 0x55afa8453300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6a35a640 1 -- 172.21.15.103:0/3117148697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d52e0 msgr2=0x55afa83e33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6a35a640 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d52e0 0x55afa83e33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6a35a640 1 -- 172.21.15.103:0/3117148697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa83e3a40 con 0x55afa83d49a0 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff69b59640 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d52e0 0x55afa83e33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6a35a640 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d49a0 0x55afa83e2e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7eff60009fd0 tx=0x7eff60005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cb817e021703e83 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:09:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff4dffb640 1 -- 172.21.15.103:0/3117148697 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600cfe60 con 0x55afa83d49a0 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6f430740 1 -- 172.21.15.103:0/3117148697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55afa833fd70 con 0x55afa83d49a0 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff4dffb640 1 -- 172.21.15.103:0/3117148697 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7eff600a4640 con 0x55afa83d49a0 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff4dffb640 1 -- 172.21.15.103:0/3117148697 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff600a4ad0 con 0x55afa83d49a0 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6f430740 10 client.25595 fetch_fsmap learned FSMap version 35 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.734+0000 7eff6f430740 1 -- 172.21.15.103:0/3117148697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55afa84549f0 con 0x55afa83d49a0 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff4dffb640 1 -- 172.21.15.103:0/3117148697 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55afa833fd70 con 0x55afa83d49a0 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff4dffb640 1 -- 172.21.15.103:0/3117148697 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7eff600baaa0 con 0x55afa83d49a0 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff6f430740 10 client.25595 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff6f430740 10 client.25595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff6f430740 20 client.25595 populate_metadata read hostname 'smithi103' 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff6f430740 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55afa83ea640 0x55afa8457350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff6f430740 4 client.25595 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:09:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff6f430740 1 -- 172.21.15.103:0/3117148697 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afa82bc900 con 0x55afa83ea640 2024-09-05T02:09:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff69b59640 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55afa83ea640 0x55afa8457350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff4dffb640 10 client.25595 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.735+0000 7eff69b59640 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55afa83ea640 0x55afa8457350 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d17f40a0aecff7 server_cookie=ebb5f45a3ea6b5a in_seq=0 out_seq=0 2024-09-05T02:09:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.848+0000 7eff4dffb640 1 -- 172.21.15.103:0/3117148697 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afa82bc900 con 0x55afa83ea640 2024-09-05T02:09:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.848+0000 7eff4dffb640 10 client.25595 handle_command_reply: tid=0 2024-09-05T02:09:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 client.25595 shutdown 2024-09-05T02:09:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 -- 172.21.15.103:0/3117148697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d49a0 msgr2=0x55afa83e2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d49a0 0x55afa83e2e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7eff60009fd0 tx=0x7eff60005cc0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 -- 172.21.15.103:0/3117148697 shutdown_connections 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55afa83ea640 0x55afa8457350 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afa83da370 0x55afa8453300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55afa83d52e0 0x55afa83e33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 --2- 172.21.15.103:0/3117148697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa83d49a0 0x55afa83e2e60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff4dffb640 0 client.25595 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 -- 172.21.15.103:0/3117148697 >> 172.21.15.103:0/3117148697 conn(0x55afa83d0010 msgr2=0x55afa83cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 -- 172.21.15.103:0/3117148697 shutdown_connections 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 1 -- 172.21.15.103:0/3117148697 wait complete. 2024-09-05T02:09:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6f430740 20 client.25595 trim_cache size 0 max 16384 2024-09-05T02:09:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:09:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:09:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:09:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6c5be640 1 -- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7eff500a2c30 msgr2=0x7eff500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6c5be640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7eff500a2c30 0x7eff500a50f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009430 tx=0x7eff5c02a8c0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6c5be640 1 -- 172.21.15.103:0/1069679557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64101b60 msgr2=0x7eff64197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.849+0000 7eff6c5be640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64101b60 0x7eff64197d00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9c0 tx=0x7eff6000ee80 comp rx=0 tx=0).stop 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.850+0000 7eff6c5be640 1 -- 172.21.15.103:0/1069679557 shutdown_connections 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.850+0000 7eff6c5be640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff64103710 0x7eff6419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.850+0000 7eff6c5be640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff64102d50 0x7eff64198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.850+0000 7eff6c5be640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7eff500a2c30 0x7eff500a50f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.850+0000 7eff6c5be640 1 --2- 172.21.15.103:0/1069679557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64101b60 0x7eff64197d00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.850+0000 7eff6c5be640 1 -- 172.21.15.103:0/1069679557 >> 172.21.15.103:0/1069679557 conn(0x7eff640fd2e0 msgr2=0x7eff64106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.850+0000 7eff6c5be640 1 -- 172.21.15.103:0/1069679557 shutdown_connections 2024-09-05T02:09:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:07.850+0000 7eff6c5be640 1 -- 172.21.15.103:0/1069679557 wait complete. 2024-09-05T02:09:08.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:08 smithi167 ceph-mon[35122]: pgmap v551: 129 pgs: 129 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-09-05T02:09:08.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:08 smithi139 ceph-mon[34262]: pgmap v551: 129 pgs: 129 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-09-05T02:09:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:08 smithi103 ceph-mon[30902]: pgmap v551: 129 pgs: 129 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-09-05T02:09:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:10 smithi167 ceph-mon[35122]: pgmap v552: 129 pgs: 129 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-09-05T02:09:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:10 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:10 smithi139 ceph-mon[34262]: pgmap v552: 129 pgs: 129 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-09-05T02:09:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:10 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:10 smithi103 ceph-mon[30902]: pgmap v552: 129 pgs: 129 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-09-05T02:09:10.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:10 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:12 smithi167 ceph-mon[35122]: pgmap v553: 129 pgs: 129 active+clean; 386 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-09-05T02:09:12.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:12 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:12 smithi139 ceph-mon[34262]: pgmap v553: 129 pgs: 129 active+clean; 386 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-09-05T02:09:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:12 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:12 smithi103 ceph-mon[30902]: pgmap v553: 129 pgs: 129 active+clean; 386 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-09-05T02:09:12.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:12 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:13.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:13 smithi167 ceph-mon[35122]: pgmap v554: 129 pgs: 129 active+clean; 382 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 267 op/s 2024-09-05T02:09:13.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:13 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:13.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:13 smithi139 ceph-mon[34262]: pgmap v554: 129 pgs: 129 active+clean; 382 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 267 op/s 2024-09-05T02:09:13.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:13 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:13.901 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:13 smithi103 ceph-mon[30902]: pgmap v554: 129 pgs: 129 active+clean; 382 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 267 op/s 2024-09-05T02:09:13.901 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:13 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:15.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:15 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:15.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:15 smithi167 ceph-mon[35122]: pgmap v555: 129 pgs: 129 active+clean; 362 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 380 op/s 2024-09-05T02:09:15.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:15 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:15.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:15 smithi139 ceph-mon[34262]: pgmap v555: 129 pgs: 129 active+clean; 362 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 380 op/s 2024-09-05T02:09:15.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:15 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:15.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:15 smithi103 ceph-mon[30902]: pgmap v555: 129 pgs: 129 active+clean; 362 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 380 op/s 2024-09-05T02:09:17.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:17 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:17.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:17 smithi167 ceph-mon[35122]: pgmap v556: 129 pgs: 129 active+clean; 359 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.0 MiB/s wr, 343 op/s 2024-09-05T02:09:17.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:17 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:17.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:17 smithi139 ceph-mon[34262]: pgmap v556: 129 pgs: 129 active+clean; 359 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.0 MiB/s wr, 343 op/s 2024-09-05T02:09:17.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:17 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:17.898 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:17 smithi103 ceph-mon[30902]: pgmap v556: 129 pgs: 129 active+clean; 359 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.0 MiB/s wr, 343 op/s 2024-09-05T02:09:19.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:19.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:09:19.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi139 to 176.0M 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: Unable to set osd_memory_target on smithi139 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: Unable to set osd_memory_target on smithi167 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-05T02:09:19.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: Unable to set osd_memory_target on smithi103 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-05T02:09:19.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:09:19.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:09:19.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:09:19.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:19 smithi167 ceph-mon[35122]: pgmap v557: 129 pgs: 129 active+clean; 353 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-09-05T02:09:19.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:19.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:09:19.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi139 to 176.0M 2024-09-05T02:09:19.896 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: Unable to set osd_memory_target on smithi139 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-05T02:09:19.897 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: Unable to set osd_memory_target on smithi167 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-05T02:09:19.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: Unable to set osd_memory_target on smithi103 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-05T02:09:19.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:09:19.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:09:19.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:09:19.898 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:19 smithi139 ceph-mon[34262]: pgmap v557: 129 pgs: 129 active+clean; 353 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-09-05T02:09:19.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:19.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:09:19.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi139 to 176.0M 2024-09-05T02:09:19.899 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: Unable to set osd_memory_target on smithi139 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: Unable to set osd_memory_target on smithi167 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: Unable to set osd_memory_target on smithi103 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-05T02:09:19.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:09:19.901 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:09:19.901 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:09:19.901 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:19 smithi103 ceph-mon[30902]: pgmap v557: 129 pgs: 129 active+clean; 353 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-09-05T02:09:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/176121299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b88100a40 msgr2=0x7f1b88100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/176121299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b88100a40 0x7f1b88100e40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009900 tx=0x7f1b7802a410 comp rx=0 tx=0).stop 2024-09-05T02:09:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/176121299 shutdown_connections 2024-09-05T02:09:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/176121299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b881025f0 0x7f1b88108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/176121299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b88101c30 0x7f1b881020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/176121299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b88100a40 0x7f1b88100e40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/176121299 >> 172.21.15.103:0/176121299 conn(0x7f1b880fc1e0 msgr2=0x7f1b880fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/176121299 shutdown_connections 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/176121299 wait complete. 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.939+0000 7f1b8d2e4640 1 Processor -- start 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b8d2e4640 1 -- start start 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b8d2e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b88100a40 0x7f1b88197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b8d2e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b88101c30 0x7f1b88198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b8d2e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b881025f0 0x7f1b8819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b8d2e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8810d6c0 con 0x7f1b88101c30 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b8d2e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8810d540 con 0x7f1b88100a40 2024-09-05T02:09:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b8d2e4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8810d840 con 0x7f1b881025f0 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b88100a40 0x7f1b88197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b867fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b88101c30 0x7f1b88198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b88100a40 0x7f1b88197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:44150/0 (socket says 172.21.15.103:44150) 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b86ffd640 1 -- 172.21.15.103:0/687730709 learned_addr learned my addr 172.21.15.103:0/687730709 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b877fe640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b881025f0 0x7f1b8819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b867fc640 1 -- 172.21.15.103:0/687730709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b881025f0 msgr2=0x7f1b8819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b867fc640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b881025f0 0x7f1b8819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b867fc640 1 -- 172.21.15.103:0/687730709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b88100a40 msgr2=0x7f1b88197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b867fc640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b88100a40 0x7f1b88197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b867fc640 1 -- 172.21.15.103:0/687730709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8819f910 con 0x7f1b88101c30 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.940+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b88100a40 0x7f1b88197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.941+0000 7f1b867fc640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b88101c30 0x7f1b88198240 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009d20 tx=0x7f1b7c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.941+0000 7f1b6ffff640 1 -- 172.21.15.103:0/687730709 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c01f070 con 0x7f1b88101c30 2024-09-05T02:09:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.941+0000 7f1b6ffff640 1 -- 172.21.15.103:0/687730709 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b7c011750 con 0x7f1b88101c30 2024-09-05T02:09:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.941+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/687730709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b881098a0 con 0x7f1b88101c30 2024-09-05T02:09:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.941+0000 7f1b6ffff640 1 -- 172.21.15.103:0/687730709 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c01a720 con 0x7f1b88101c30 2024-09-05T02:09:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.941+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/687730709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b88109e40 con 0x7f1b88101c30 2024-09-05T02:09:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.943+0000 7f1b6ffff640 1 -- 172.21.15.103:0/687730709 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1b7c022070 con 0x7f1b88101c30 2024-09-05T02:09:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b90085740 1 Processor -- start 2024-09-05T02:09:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b90085740 1 -- start start 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b90085740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b609da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b90085740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b60eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b90085740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 0x56274b611830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b90085740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56274b56d2f0 con 0x56274b6099a0 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b90085740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56274b515270 con 0x56274b60f370 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b90085740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56274b5692b0 con 0x56274b60a2e0 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b6ffff640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b700a2cb0 0x7f1b700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b60eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b609da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b60eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.103:36478/0 (socket says 172.21.15.103:36478) 2024-09-05T02:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b86ffd640 1 -- 172.21.15.103:0/3568021712 learned_addr learned my addr 172.21.15.103:0/3568021712 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b867fc640 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 0x56274b611830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.944+0000 7f1b877fe640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b700a2cb0 0x7f1b700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b6ffff640 1 -- 172.21.15.103:0/687730709 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1b7c0d3040 con 0x7f1b88101c30 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b86ffd640 1 -- 172.21.15.103:0/3568021712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 msgr2=0x56274b611830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 0x56274b611830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b86ffd640 1 -- 172.21.15.103:0/3568021712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 msgr2=0x56274b609da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b609da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b86ffd640 1 -- 172.21.15.103:0/3568021712 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56274b514340 con 0x56274b60a2e0 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b877fe640 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b609da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b867fc640 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 0x56274b611830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b877fe640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b700a2cb0 0x7f1b700a5170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1b80005c40 tx=0x7f1b80009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b60eda0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009230 tx=0x7f1b78005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f9425442fe69e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b6d7fa640 1 -- 172.21.15.103:0/3568021712 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78035070 con 0x56274b60a2e0 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b6d7fa640 1 -- 172.21.15.103:0/3568021712 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b78031800 con 0x56274b60a2e0 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.945+0000 7f1b6d7fa640 1 -- 172.21.15.103:0/3568021712 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7803c440 con 0x56274b60a2e0 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 -- 172.21.15.103:0/3568021712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 msgr2=0x56274b60eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b60eda0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009230 tx=0x7f1b78005c00 comp rx=0 tx=0).stop 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 -- 172.21.15.103:0/3568021712 shutdown_connections 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 0x56274b611830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b60eda0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 --2- 172.21.15.103:0/3568021712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b609da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 -- 172.21.15.103:0/3568021712 >> 172.21.15.103:0/3568021712 conn(0x56274b605010 msgr2=0x56274b605410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 -- 172.21.15.103:0/3568021712 shutdown_connections 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 -- 172.21.15.103:0/3568021712 wait complete. 2024-09-05T02:09:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.946+0000 7f1b90085740 1 -- start start 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b90085740 1 Processor -- start 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b90085740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b613b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b90085740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60a2e0 0x56274b6140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b90085740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60f370 0x56274b61b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b90085740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56274b6122e0 con 0x56274b6099a0 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b90085740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56274b4f6290 con 0x56274b60a2e0 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b90085740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56274b612460 con 0x56274b60f370 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b90085740 10 client.0 _ll_register_callbacks cb 0x56274b527140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b613b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b613b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49348/0 (socket says 172.21.15.103:49348) 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60f370 0x56274b61b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b86ffd640 1 -- 172.21.15.103:0/2635757420 learned_addr learned my addr 172.21.15.103:0/2635757420 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b867fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60a2e0 0x56274b6140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b86ffd640 1 -- 172.21.15.103:0/2635757420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60f370 msgr2=0x56274b61b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60f370 0x56274b61b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b86ffd640 1 -- 172.21.15.103:0/2635757420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60a2e0 msgr2=0x56274b6140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60a2e0 0x56274b6140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b86ffd640 1 -- 172.21.15.103:0/2635757420 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56274b280210 con 0x56274b6099a0 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b613b70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1b780314e0 tx=0x7f1b78009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d314d29aabcd592a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.947+0000 7f1b6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b6aff5640 1 -- 172.21.15.103:0/2635757420 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7804e020 con 0x56274b6099a0 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b6aff5640 1 -- 172.21.15.103:0/2635757420 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1b78003e00 con 0x56274b6099a0 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b90085740 1 -- 172.21.15.103:0/2635757420 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b78031800 con 0x56274b6099a0 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b6aff5640 1 -- 172.21.15.103:0/2635757420 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1b78059050 con 0x56274b6099a0 2024-09-05T02:09:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b90085740 10 client.16212 fetch_fsmap learned FSMap version 41 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b90085740 1 -- 172.21.15.103:0/2635757420 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56274b34c620 con 0x56274b6099a0 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b6aff5640 1 -- 172.21.15.103:0/2635757420 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b78031800 con 0x56274b6099a0 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b6aff5640 1 -- 172.21.15.103:0/2635757420 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 41) ==== 8442+0+0 (secure 0 0 0) 0x7f1b78045a10 con 0x56274b6099a0 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b90085740 10 client.16212 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b90085740 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b90085740 20 client.16212 populate_metadata read hostname 'smithi103' 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b90085740 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56274b61f440 0x56274b621830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b90085740 4 client.16212 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b90085740 1 -- 172.21.15.103:0/2635757420 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56274b57aa20 con 0x56274b61f440 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.948+0000 7f1b867fc640 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56274b61f440 0x56274b621830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.949+0000 7f1b6aff5640 10 client.16212 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.949+0000 7f1b867fc640 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56274b61f440 0x56274b621830 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8a8074a9c7d06f server_cookie=4702758f48df77c5 in_seq=0 out_seq=0 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.949+0000 7f1b6aff5640 1 -- 172.21.15.103:0/2635757420 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56274b57aa20 con 0x56274b61f440 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.949+0000 7f1b6aff5640 10 client.16212 handle_command_reply: tid=0 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.949+0000 7f1b90085740 1 client.16212 shutdown 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 -- 172.21.15.103:0/2635757420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 msgr2=0x56274b613b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b613b70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1b780314e0 tx=0x7f1b78009520 comp rx=0 tx=0).stop 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 -- 172.21.15.103:0/2635757420 shutdown_connections 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56274b61f440 0x56274b621830 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60f370 0x56274b61b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60a2e0 0x56274b6140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b6aff5640 0 client.16212 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 --2- 172.21.15.103:0/2635757420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b6099a0 0x56274b613b70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 -- 172.21.15.103:0/2635757420 >> 172.21.15.103:0/2635757420 conn(0x56274b605010 msgr2=0x56274b606820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 -- 172.21.15.103:0/2635757420 shutdown_connections 2024-09-05T02:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 1 -- 172.21.15.103:0/2635757420 wait complete. 2024-09-05T02:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.950+0000 7f1b90085740 20 client.16212 trim_cache size 0 max 16384 2024-09-05T02:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b90085740 1 Processor -- start 2024-09-05T02:09:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b90085740 1 -- start start 2024-09-05T02:09:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b90085740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b6099a0 0x56274b60eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b90085740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b61b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b90085740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60f370 0x56274b606400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b90085740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b881a83a0 con 0x56274b60f370 2024-09-05T02:09:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b90085740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b881aa180 con 0x56274b6099a0 2024-09-05T02:09:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b90085740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b881a8b30 con 0x56274b60a2e0 2024-09-05T02:09:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60f370 0x56274b606400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b867fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b61b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60f370 0x56274b606400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49358/0 (socket says 172.21.15.103:49358) 2024-09-05T02:09:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b877fe640 1 -- 172.21.15.103:0/2647782233 learned_addr learned my addr 172.21.15.103:0/2647782233 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/2647782233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b6099a0 0x56274b60eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b867fc640 1 -- 172.21.15.103:0/2647782233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b6099a0 msgr2=0x56274b60eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b867fc640 1 --2- 172.21.15.103:0/2647782233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b6099a0 0x56274b60eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b867fc640 1 -- 172.21.15.103:0/2647782233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60f370 msgr2=0x56274b606400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b867fc640 1 --2- 172.21.15.103:0/2647782233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60f370 0x56274b606400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b867fc640 1 -- 172.21.15.103:0/2647782233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56274b614000 con 0x56274b60a2e0 2024-09-05T02:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.965+0000 7f1b877fe640 1 --2- 172.21.15.103:0/2647782233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60f370 0x56274b606400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b867fc640 1 --2- 172.21.15.103:0/2647782233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b61b0b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0d07e0 tx=0x7f1b7c071520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf1db138b070dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b6cff9640 1 -- 172.21.15.103:0/2647782233 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0a4a30 con 0x56274b60a2e0 2024-09-05T02:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b6cff9640 1 -- 172.21.15.103:0/2647782233 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b7c011c20 con 0x56274b60a2e0 2024-09-05T02:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b6cff9640 1 -- 172.21.15.103:0/2647782233 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c09d970 con 0x56274b60a2e0 2024-09-05T02:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b90085740 1 -- 172.21.15.103:0/2647782233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 msgr2=0x56274b61b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b90085740 1 --2- 172.21.15.103:0/2647782233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b61b0b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0d07e0 tx=0x7f1b7c071520 comp rx=0 tx=0).stop 2024-09-05T02:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b90085740 1 -- 172.21.15.103:0/2647782233 shutdown_connections 2024-09-05T02:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b90085740 1 --2- 172.21.15.103:0/2647782233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60f370 0x56274b606400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b90085740 1 --2- 172.21.15.103:0/2647782233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b60a2e0 0x56274b61b0b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b90085740 1 --2- 172.21.15.103:0/2647782233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b6099a0 0x56274b60eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b90085740 1 -- 172.21.15.103:0/2647782233 >> 172.21.15.103:0/2647782233 conn(0x56274b605010 msgr2=0x56274b6041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b90085740 1 -- 172.21.15.103:0/2647782233 shutdown_connections 2024-09-05T02:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.966+0000 7f1b90085740 1 -- 172.21.15.103:0/2647782233 wait complete. 2024-09-05T02:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b90085740 1 -- start start 2024-09-05T02:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b90085740 1 Processor -- start 2024-09-05T02:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b90085740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b6099a0 0x56274b618c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b90085740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60a2e0 0x56274b6191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b90085740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 0x56274b61a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b90085740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b881a9a50 con 0x56274b60a2e0 2024-09-05T02:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b90085740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b881a92c0 con 0x56274b60f370 2024-09-05T02:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b90085740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b881a6380 con 0x56274b6099a0 2024-09-05T02:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b90085740 10 client.0 _ll_register_callbacks cb 0x7f1b7c09b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b867fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60a2e0 0x56274b6191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b867fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60a2e0 0x56274b6191b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49368/0 (socket says 172.21.15.103:49368) 2024-09-05T02:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b867fc640 1 -- 172.21.15.103:0/3773212298 learned_addr learned my addr 172.21.15.103:0/3773212298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b877fe640 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 0x56274b61a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b6099a0 0x56274b618c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b867fc640 1 -- 172.21.15.103:0/3773212298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b6099a0 msgr2=0x56274b618c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b867fc640 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b6099a0 0x56274b618c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b867fc640 1 -- 172.21.15.103:0/3773212298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 msgr2=0x56274b61a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b867fc640 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 0x56274b61a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b867fc640 1 -- 172.21.15.103:0/3773212298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56274b621ba0 con 0x56274b60a2e0 2024-09-05T02:09:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.967+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b6099a0 0x56274b618c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b867fc640 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60a2e0 0x56274b6191b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c09b970 tx=0x7f1b7c0a4c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342865edbfd6f999 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b6aff5640 1 -- 172.21.15.103:0/3773212298 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c011900 con 0x56274b60a2e0 2024-09-05T02:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b6aff5640 1 -- 172.21.15.103:0/3773212298 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1b7c09bb80 con 0x56274b60a2e0 2024-09-05T02:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b90085740 1 -- 172.21.15.103:0/3773212298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56274b574d70 con 0x56274b60a2e0 2024-09-05T02:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b6aff5640 1 -- 172.21.15.103:0/3773212298 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1b7c0a7cd0 con 0x56274b60a2e0 2024-09-05T02:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b90085740 10 client.16218 fetch_fsmap learned FSMap version 41 2024-09-05T02:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b90085740 1 -- 172.21.15.103:0/3773212298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56274b68a3d0 con 0x56274b60a2e0 2024-09-05T02:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b6aff5640 1 -- 172.21.15.103:0/3773212298 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56274b574d70 con 0x56274b60a2e0 2024-09-05T02:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b6aff5640 1 -- 172.21.15.103:0/3773212298 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 41) ==== 8442+0+0 (secure 0 0 0) 0x7f1b7c09e9f0 con 0x56274b60a2e0 2024-09-05T02:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b90085740 10 client.16218 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T02:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b90085740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:09:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b90085740 20 client.16218 populate_metadata read hostname 'smithi103' 2024-09-05T02:09:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b90085740 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56274b68ad90 0x56274b68d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b90085740 4 client.16218 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:09:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.968+0000 7f1b90085740 1 -- 172.21.15.103:0/3773212298 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56274b4f1900 con 0x56274b68ad90 2024-09-05T02:09:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.969+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56274b68ad90 0x56274b68d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.969+0000 7f1b86ffd640 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56274b68ad90 0x56274b68d1b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e393fe860409e04f server_cookie=3861d17c5a7a009f in_seq=0 out_seq=0 2024-09-05T02:09:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:20.969+0000 7f1b6aff5640 10 client.16218 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.001+0000 7f1b6aff5640 1 -- 172.21.15.103:0/3773212298 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56274b4f1900 con 0x56274b68ad90 2024-09-05T02:09:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.001+0000 7f1b6aff5640 10 client.16218 handle_command_reply: tid=0 2024-09-05T02:09:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.001+0000 7f1b90085740 1 client.16218 shutdown 2024-09-05T02:09:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.001+0000 7f1b90085740 1 -- 172.21.15.103:0/3773212298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60a2e0 msgr2=0x56274b6191b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.001+0000 7f1b90085740 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60a2e0 0x56274b6191b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c09b970 tx=0x7f1b7c0a4c80 comp rx=0 tx=0).stop 2024-09-05T02:09:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b90085740 1 -- 172.21.15.103:0/3773212298 shutdown_connections 2024-09-05T02:09:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b90085740 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x56274b68ad90 0x56274b68d1b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b90085740 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56274b60f370 0x56274b61a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b90085740 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56274b60a2e0 0x56274b6191b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b6aff5640 0 client.16218 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b90085740 1 --2- 172.21.15.103:0/3773212298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274b6099a0 0x56274b618c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b90085740 1 -- 172.21.15.103:0/3773212298 >> 172.21.15.103:0/3773212298 conn(0x56274b605010 msgr2=0x56274b6041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b90085740 1 -- 172.21.15.103:0/3773212298 shutdown_connections 2024-09-05T02:09:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b90085740 1 -- 172.21.15.103:0/3773212298 wait complete. 2024-09-05T02:09:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b90085740 20 client.16218 trim_cache size 0 max 16384 2024-09-05T02:09:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:09:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:09:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:09:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:09:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b700a2cb0 msgr2=0x7f1b700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b700a2cb0 0x7f1b700a5170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1b80005c40 tx=0x7f1b80009500 comp rx=0 tx=0).stop 2024-09-05T02:09:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b88101c30 msgr2=0x7f1b88198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b88101c30 0x7f1b88198240 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009d20 tx=0x7f1b7c00c6a0 comp rx=0 tx=0).stop 2024-09-05T02:09:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/687730709 shutdown_connections 2024-09-05T02:09:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b700a2cb0 0x7f1b700a5170 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b881025f0 0x7f1b8819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b88101c30 0x7f1b88198240 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 --2- 172.21.15.103:0/687730709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b88100a40 0x7f1b88197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.002+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/687730709 >> 172.21.15.103:0/687730709 conn(0x7f1b880fc1e0 msgr2=0x7f1b88105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.003+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/687730709 shutdown_connections 2024-09-05T02:09:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:21.003+0000 7f1b8d2e4640 1 -- 172.21.15.103:0/687730709 wait complete. 2024-09-05T02:09:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:22 smithi167 ceph-mon[35122]: pgmap v558: 129 pgs: 129 active+clean; 340 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 976 KiB/s wr, 391 op/s 2024-09-05T02:09:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:22 smithi139 ceph-mon[34262]: pgmap v558: 129 pgs: 129 active+clean; 340 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 976 KiB/s wr, 391 op/s 2024-09-05T02:09:22.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:22 smithi103 ceph-mon[30902]: pgmap v558: 129 pgs: 129 active+clean; 340 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 976 KiB/s wr, 391 op/s 2024-09-05T02:09:24.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:24 smithi167 ceph-mon[35122]: pgmap v559: 129 pgs: 129 active+clean; 334 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 881 KiB/s wr, 373 op/s 2024-09-05T02:09:24.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:24 smithi139 ceph-mon[34262]: pgmap v559: 129 pgs: 129 active+clean; 334 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 881 KiB/s wr, 373 op/s 2024-09-05T02:09:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:24 smithi103 ceph-mon[30902]: pgmap v559: 129 pgs: 129 active+clean; 334 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 881 KiB/s wr, 373 op/s 2024-09-05T02:09:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:26 smithi167 ceph-mon[35122]: pgmap v560: 129 pgs: 129 active+clean; 318 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 443 op/s 2024-09-05T02:09:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:26 smithi139 ceph-mon[34262]: pgmap v560: 129 pgs: 129 active+clean; 318 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 443 op/s 2024-09-05T02:09:26.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:26 smithi103 ceph-mon[30902]: pgmap v560: 129 pgs: 129 active+clean; 318 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 443 op/s 2024-09-05T02:09:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:28 smithi167 ceph-mon[35122]: pgmap v561: 129 pgs: 129 active+clean; 317 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 932 KiB/s wr, 338 op/s 2024-09-05T02:09:28.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:28 smithi139 ceph-mon[34262]: pgmap v561: 129 pgs: 129 active+clean; 317 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 932 KiB/s wr, 338 op/s 2024-09-05T02:09:28.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:28 smithi103 ceph-mon[30902]: pgmap v561: 129 pgs: 129 active+clean; 317 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 932 KiB/s wr, 338 op/s 2024-09-05T02:09:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:30 smithi167 ceph-mon[35122]: pgmap v562: 129 pgs: 129 active+clean; 308 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 405 op/s 2024-09-05T02:09:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:30 smithi167 ceph-mon[35122]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:30.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:30 smithi139 ceph-mon[34262]: pgmap v562: 129 pgs: 129 active+clean; 308 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 405 op/s 2024-09-05T02:09:30.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:30 smithi139 ceph-mon[34262]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:30 smithi103 ceph-mon[30902]: pgmap v562: 129 pgs: 129 active+clean; 308 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 405 op/s 2024-09-05T02:09:30.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:30 smithi103 ceph-mon[30902]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T02:09:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:32 smithi167 ceph-mon[35122]: pgmap v563: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 901 KiB/s wr, 442 op/s 2024-09-05T02:09:32.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:32 smithi139 ceph-mon[34262]: pgmap v563: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 901 KiB/s wr, 442 op/s 2024-09-05T02:09:32.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:32 smithi103 ceph-mon[30902]: pgmap v563: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 901 KiB/s wr, 442 op/s 2024-09-05T02:09:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/2968552330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 msgr2=0x7f1f48101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/2968552330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 0x7f1f48101f60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009900 tx=0x7f1f3802a410 comp rx=0 tx=0).stop 2024-09-05T02:09:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/2968552330 shutdown_connections 2024-09-05T02:09:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/2968552330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f48103710 0x7f1f48109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/2968552330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48102d50 0x7f1f481031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/2968552330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 0x7f1f48101f60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/2968552330 >> 172.21.15.103:0/2968552330 conn(0x7f1f480fd2c0 msgr2=0x7f1f480ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/2968552330 shutdown_connections 2024-09-05T02:09:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/2968552330 wait complete. 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.104+0000 7f1f4e0b7640 1 Processor -- start 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f4e0b7640 1 -- start start 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f4e0b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 0x7f1f48197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f4e0b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48102d50 0x7f1f481982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f4e0b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f48103710 0x7f1f4819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f4e0b7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810d690 con 0x7f1f48101b60 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f4e0b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810d510 con 0x7f1f48102d50 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f4e0b7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810d810 con 0x7f1f48103710 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 0x7f1f48197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 0x7f1f48197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49292/0 (socket says 172.21.15.103:49292) 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f477fe640 1 -- 172.21.15.103:0/443269358 learned_addr learned my addr 172.21.15.103:0/443269358 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48102d50 0x7f1f481982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f47fff640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f48103710 0x7f1f4819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f477fe640 1 -- 172.21.15.103:0/443269358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f48103710 msgr2=0x7f1f4819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f477fe640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f48103710 0x7f1f4819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f477fe640 1 -- 172.21.15.103:0/443269358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48102d50 msgr2=0x7f1f481982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f477fe640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48102d50 0x7f1f481982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f477fe640 1 -- 172.21.15.103:0/443269358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4819f990 con 0x7f1f48101b60 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.105+0000 7f1f477fe640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 0x7f1f48197d70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1f38004270 tx=0x7f1f380042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.106+0000 7f1f44ff9640 1 -- 172.21.15.103:0/443269358 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3803f030 con 0x7f1f48101b60 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.106+0000 7f1f44ff9640 1 -- 172.21.15.103:0/443269358 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f380318d0 con 0x7f1f48101b60 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.106+0000 7f1f44ff9640 1 -- 172.21.15.103:0/443269358 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38035070 con 0x7f1f48101b60 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.106+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/443269358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4819fc20 con 0x7f1f48101b60 2024-09-05T02:09:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.106+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/443269358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481a0110 con 0x7f1f48101b60 2024-09-05T02:09:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.108+0000 7f1f44ff9640 1 -- 172.21.15.103:0/443269358 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1f38042030 con 0x7f1f48101b60 2024-09-05T02:09:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.108+0000 7f1f50e50740 1 Processor -- start 2024-09-05T02:09:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.108+0000 7f1f50e50740 1 -- start start 2024-09-05T02:09:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 0x55cdbb620da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6212e0 0x55cdbb625da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb628830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f50e50740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cdbb5842f0 con 0x55cdbb626370 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f50e50740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cdbb52c270 con 0x55cdbb6212e0 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f50e50740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cdbb5802b0 con 0x55cdbb6209a0 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb628830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f44ff9640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f300a2c30 0x7f1f300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6212e0 0x55cdbb625da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb628830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49294/0 (socket says 172.21.15.103:49294) 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f46ffd640 1 -- 172.21.15.103:0/3738405620 learned_addr learned my addr 172.21.15.103:0/3738405620 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f47fff640 1 --2- 172.21.15.103:0/3738405620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 0x55cdbb620da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f47fff640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f300a2c30 0x7f1f300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f46ffd640 1 -- 172.21.15.103:0/3738405620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 msgr2=0x55cdbb620da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f44ff9640 1 -- 172.21.15.103:0/443269358 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f380ef7e0 con 0x7f1f48101b60 2024-09-05T02:09:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/3738405620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 0x55cdbb620da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f46ffd640 1 -- 172.21.15.103:0/3738405620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6212e0 msgr2=0x55cdbb625da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/3738405620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6212e0 0x55cdbb625da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f46ffd640 1 -- 172.21.15.103:0/3738405620 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdbb52b340 con 0x55cdbb626370 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.109+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/3738405620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb628830 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009c90 tx=0x7f1f3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef966dcfbbb0b45f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f47fff640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f300a2c30 0x7f1f300a50f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f40005c40 tx=0x7f1f40009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f2dffb640 1 -- 172.21.15.103:0/3738405620 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c01e070 con 0x55cdbb626370 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f2dffb640 1 -- 172.21.15.103:0/3738405620 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f3c019ae0 con 0x55cdbb626370 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f2dffb640 1 -- 172.21.15.103:0/3738405620 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00e960 con 0x55cdbb626370 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f50e50740 1 -- 172.21.15.103:0/3738405620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 msgr2=0x55cdbb628830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3738405620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb628830 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009c90 tx=0x7f1f3c011810 comp rx=0 tx=0).stop 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f50e50740 1 -- 172.21.15.103:0/3738405620 shutdown_connections 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3738405620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb628830 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3738405620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6212e0 0x55cdbb625da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3738405620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 0x55cdbb620da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f50e50740 1 -- 172.21.15.103:0/3738405620 >> 172.21.15.103:0/3738405620 conn(0x55cdbb61c010 msgr2=0x55cdbb61c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f50e50740 1 -- 172.21.15.103:0/3738405620 shutdown_connections 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.110+0000 7f1f50e50740 1 -- 172.21.15.103:0/3738405620 wait complete. 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f50e50740 1 -- start start 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f50e50740 1 Processor -- start 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6212e0 0x55cdbb62b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb632110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f50e50740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cdbb629340 con 0x55cdbb626370 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f50e50740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cdbb50d290 con 0x55cdbb6209a0 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f50e50740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cdbb6294c0 con 0x55cdbb6212e0 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f50e50740 10 client.0 _ll_register_callbacks cb 0x55cdbb53e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:09:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb632110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb632110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49304/0 (socket says 172.21.15.103:49304) 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f47fff640 1 -- 172.21.15.103:0/3568566178 learned_addr learned my addr 172.21.15.103:0/3568566178 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.111+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6212e0 0x55cdbb62b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f47fff640 1 -- 172.21.15.103:0/3568566178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6212e0 msgr2=0x55cdbb62b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f47fff640 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6212e0 0x55cdbb62b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f47fff640 1 -- 172.21.15.103:0/3568566178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 msgr2=0x55cdbb62ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f47fff640 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f47fff640 1 -- 172.21.15.103:0/3568566178 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdbb297210 con 0x55cdbb626370 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6212e0 0x55cdbb62b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f477fe640 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f47fff640 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb632110 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000fb30 tx=0x7f1f40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae9802924155a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:09:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f26ffd640 1 -- 172.21.15.103:0/3568566178 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40011020 con 0x55cdbb626370 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f26ffd640 1 -- 172.21.15.103:0/3568566178 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f400111c0 con 0x55cdbb626370 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f50e50740 1 -- 172.21.15.103:0/3568566178 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3c019ae0 con 0x55cdbb626370 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f26ffd640 1 -- 172.21.15.103:0/3568566178 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1f40004260 con 0x55cdbb626370 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f50e50740 10 client.16230 fetch_fsmap learned FSMap version 42 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f50e50740 1 -- 172.21.15.103:0/3568566178 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cdbb363620 con 0x55cdbb626370 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.112+0000 7f1f26ffd640 1 -- 172.21.15.103:0/3568566178 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f3c019ae0 con 0x55cdbb626370 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f26ffd640 1 -- 172.21.15.103:0/3568566178 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 42) ==== 8450+0+0 (secure 0 0 0) 0x7f1f40027070 con 0x55cdbb626370 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f50e50740 10 client.16230 fetch_fsmap finished waiting for FSMap version 42 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f50e50740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f50e50740 20 client.16230 populate_metadata read hostname 'smithi103' 2024-09-05T02:09:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55cdbb636dd0 0x55cdbb639270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f50e50740 4 client.16230 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f50e50740 1 -- 172.21.15.103:0/3568566178 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdbb591a20 con 0x55cdbb636dd0 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f477fe640 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55cdbb636dd0 0x55cdbb639270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f26ffd640 10 client.16230 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.113+0000 7f1f477fe640 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55cdbb636dd0 0x55cdbb639270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54fc1f6114f112cf server_cookie=1fd6e772ac8f228e in_seq=0 out_seq=0 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f26ffd640 1 -- 172.21.15.103:0/3568566178 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdbb591a20 con 0x55cdbb636dd0 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f26ffd640 10 client.16230 handle_command_reply: tid=0 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f50e50740 1 client.16230 shutdown 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f50e50740 1 -- 172.21.15.103:0/3568566178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 msgr2=0x55cdbb632110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb632110 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000fb30 tx=0x7f1f40004a40 comp rx=0 tx=0).stop 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f50e50740 1 -- 172.21.15.103:0/3568566178 shutdown_connections 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55cdbb636dd0 0x55cdbb639270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb626370 0x55cdbb632110 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6212e0 0x55cdbb62b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f26ffd640 0 client.16230 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3568566178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.114+0000 7f1f50e50740 1 -- 172.21.15.103:0/3568566178 >> 172.21.15.103:0/3568566178 conn(0x55cdbb61c010 msgr2=0x55cdbb61d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.115+0000 7f1f50e50740 1 -- 172.21.15.103:0/3568566178 shutdown_connections 2024-09-05T02:09:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.115+0000 7f1f50e50740 1 -- 172.21.15.103:0/3568566178 wait complete. 2024-09-05T02:09:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.115+0000 7f1f50e50740 20 client.16230 trim_cache size 0 max 16384 2024-09-05T02:09:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f50e50740 1 Processor -- start 2024-09-05T02:09:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f50e50740 1 -- start start 2024-09-05T02:09:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 0x55cdbb620d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb6320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb626370 0x55cdbb61cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f50e50740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481aa740 con 0x55cdbb6212e0 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f50e50740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481ac580 con 0x55cdbb626370 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f50e50740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481aaed0 con 0x55cdbb6209a0 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb6320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb6320d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49318/0 (socket says 172.21.15.103:49318) 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f46ffd640 1 -- 172.21.15.103:0/3082009115 learned_addr learned my addr 172.21.15.103:0/3082009115 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f477fe640 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 0x55cdbb620d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.130+0000 7f1f47fff640 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb626370 0x55cdbb61cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f46ffd640 1 -- 172.21.15.103:0/3082009115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 msgr2=0x55cdbb620d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 0x55cdbb620d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f46ffd640 1 -- 172.21.15.103:0/3082009115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb626370 msgr2=0x55cdbb61cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb626370 0x55cdbb61cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f46ffd640 1 -- 172.21.15.103:0/3082009115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdbb628ba0 con 0x55cdbb6212e0 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f477fe640 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 0x55cdbb620d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb6320d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55cdbb61ce90 tx=0x7f1f3c00e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4781b9040d601a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f2d7fa640 1 -- 172.21.15.103:0/3082009115 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c028070 con 0x55cdbb6212e0 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f2d7fa640 1 -- 172.21.15.103:0/3082009115 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f3c01f040 con 0x55cdbb6212e0 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.131+0000 7f1f2d7fa640 1 -- 172.21.15.103:0/3082009115 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c01e050 con 0x55cdbb6212e0 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 -- 172.21.15.103:0/3082009115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 msgr2=0x55cdbb6320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb6320d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55cdbb61ce90 tx=0x7f1f3c00e3b0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 -- 172.21.15.103:0/3082009115 shutdown_connections 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb626370 0x55cdbb61cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb6320d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 --2- 172.21.15.103:0/3082009115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb6209a0 0x55cdbb620d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 -- 172.21.15.103:0/3082009115 >> 172.21.15.103:0/3082009115 conn(0x55cdbb61c010 msgr2=0x55cdbb62ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 -- 172.21.15.103:0/3082009115 shutdown_connections 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 -- 172.21.15.103:0/3082009115 wait complete. 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 -- start start 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 Processor -- start 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb62f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.132+0000 7f1f50e50740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb626370 0x55cdbb69fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f50e50740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481abdf0 con 0x55cdbb6212e0 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f50e50740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481ab660 con 0x55cdbb6209a0 2024-09-05T02:09:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f50e50740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481a8720 con 0x55cdbb626370 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f50e50740 10 client.0 _ll_register_callbacks cb 0x7f1f40009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb62f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.103:36104/0 (socket says 172.21.15.103:36104) 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb62f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49334/0 (socket says 172.21.15.103:49334) 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f477fe640 1 -- 172.21.15.103:0/2224676393 learned_addr learned my addr 172.21.15.103:0/2224676393 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f47fff640 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb626370 0x55cdbb69fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f477fe640 1 -- 172.21.15.103:0/2224676393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb626370 msgr2=0x55cdbb69fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f477fe640 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb626370 0x55cdbb69fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f477fe640 1 -- 172.21.15.103:0/2224676393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 msgr2=0x55cdbb62f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f477fe640 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb62f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f477fe640 1 -- 172.21.15.103:0/2224676393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdbb6a0040 con 0x55cdbb6209a0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f477fe640 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62efd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f380bba40 tx=0x7f1f3804b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee6eef6ab0987afe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f26ffd640 1 -- 172.21.15.103:0/2224676393 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38049040 con 0x55cdbb6209a0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f50e50740 1 -- 172.21.15.103:0/2224676393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3c01f040 con 0x55cdbb6209a0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f26ffd640 1 -- 172.21.15.103:0/2224676393 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f380ba040 con 0x55cdbb6209a0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f26ffd640 1 -- 172.21.15.103:0/2224676393 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1f380c4900 con 0x55cdbb6209a0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.133+0000 7f1f50e50740 10 client.25631 fetch_fsmap learned FSMap version 42 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f50e50740 1 -- 172.21.15.103:0/2224676393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cdbb6a1300 con 0x55cdbb6209a0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f26ffd640 1 -- 172.21.15.103:0/2224676393 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f3c01f040 con 0x55cdbb6209a0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f26ffd640 1 -- 172.21.15.103:0/2224676393 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) ==== 8450+0+0 (secure 0 0 0) 0x7f1f380c4470 con 0x55cdbb6209a0 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f50e50740 10 client.25631 fetch_fsmap finished waiting for FSMap version 42 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f50e50740 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f50e50740 20 client.25631 populate_metadata read hostname 'smithi103' 2024-09-05T02:09:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f50e50740 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55cdbb6a1dc0 0x55cdbb6a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f50e50740 4 client.25631 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T02:09:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f50e50740 1 -- 172.21.15.103:0/2224676393 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cdbb58bd70 con 0x55cdbb6a1dc0 2024-09-05T02:09:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.134+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55cdbb6a1dc0 0x55cdbb6a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.135+0000 7f1f46ffd640 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55cdbb6a1dc0 0x55cdbb6a41b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d539dce441426d server_cookie=bc4b28b66391f696 in_seq=0 out_seq=0 2024-09-05T02:09:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.135+0000 7f1f26ffd640 10 client.25631 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f26ffd640 1 -- 172.21.15.103:0/2224676393 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cdbb58bd70 con 0x55cdbb6a1dc0 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f26ffd640 10 client.25631 handle_command_reply: tid=0 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f50e50740 1 client.25631 shutdown 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f50e50740 1 -- 172.21.15.103:0/2224676393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 msgr2=0x55cdbb62efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f50e50740 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62efd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f380bba40 tx=0x7f1f3804b000 comp rx=0 tx=0).stop 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f50e50740 1 -- 172.21.15.103:0/2224676393 shutdown_connections 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f50e50740 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55cdbb6a1dc0 0x55cdbb6a41b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f50e50740 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdbb626370 0x55cdbb69fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f50e50740 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdbb6212e0 0x55cdbb62f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f26ffd640 0 client.25631 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f50e50740 1 --2- 172.21.15.103:0/2224676393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdbb6209a0 0x55cdbb62efd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.417+0000 7f1f50e50740 1 -- 172.21.15.103:0/2224676393 >> 172.21.15.103:0/2224676393 conn(0x55cdbb61c010 msgr2=0x55cdbb61a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f50e50740 1 -- 172.21.15.103:0/2224676393 shutdown_connections 2024-09-05T02:09:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f50e50740 1 -- 172.21.15.103:0/2224676393 wait complete. 2024-09-05T02:09:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f50e50740 20 client.25631 trim_cache size 0 max 16384 2024-09-05T02:09:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-05T02:09:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-05T02:09:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-05T02:09:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-05T02:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f300a2c30 msgr2=0x7f1f300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f300a2c30 0x7f1f300a50f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f40005c40 tx=0x7f1f40009500 comp rx=0 tx=0).stop 2024-09-05T02:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 msgr2=0x7f1f48197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 0x7f1f48197d70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1f38004270 tx=0x7f1f380042a0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/443269358 shutdown_connections 2024-09-05T02:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1f300a2c30 0x7f1f300a50f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f48103710 0x7f1f4819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f48102d50 0x7f1f481982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 --2- 172.21.15.103:0/443269358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f48101b60 0x7f1f48197d70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/443269358 >> 172.21.15.103:0/443269358 conn(0x7f1f480fd2c0 msgr2=0x7f1f481061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/443269358 shutdown_connections 2024-09-05T02:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-05T02:09:34.418+0000 7f1f4e0b7640 1 -- 172.21.15.103:0/443269358 wait complete. 2024-09-05T02:09:34.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:34 smithi167 ceph-mon[35122]: pgmap v564: 129 pgs: 129 active+clean; 286 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 938 KiB/s wr, 428 op/s 2024-09-05T02:09:34.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:34 smithi139 ceph-mon[34262]: pgmap v564: 129 pgs: 129 active+clean; 286 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 938 KiB/s wr, 428 op/s 2024-09-05T02:09:34.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:34 smithi103 ceph-mon[30902]: pgmap v564: 129 pgs: 129 active+clean; 286 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 938 KiB/s wr, 428 op/s 2024-09-05T02:09:36.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:36 smithi167 ceph-mon[35122]: pgmap v565: 129 pgs: 129 active+clean; 269 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.0 MiB/s wr, 518 op/s 2024-09-05T02:09:36.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:36 smithi139 ceph-mon[34262]: pgmap v565: 129 pgs: 129 active+clean; 269 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.0 MiB/s wr, 518 op/s 2024-09-05T02:09:36.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:36 smithi103 ceph-mon[30902]: pgmap v565: 129 pgs: 129 active+clean; 269 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.0 MiB/s wr, 518 op/s 2024-09-05T02:09:38.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:38 smithi167 ceph-mon[35122]: pgmap v566: 129 pgs: 129 active+clean; 268 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 832 KiB/s wr, 431 op/s 2024-09-05T02:09:38.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:38 smithi139 ceph-mon[34262]: pgmap v566: 129 pgs: 129 active+clean; 268 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 832 KiB/s wr, 431 op/s 2024-09-05T02:09:38.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:38 smithi103 ceph-mon[30902]: pgmap v566: 129 pgs: 129 active+clean; 268 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 832 KiB/s wr, 431 op/s 2024-09-05T02:09:40.385 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-05T02:09:40.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-05T02:09:40.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:40 smithi167 ceph-mon[35122]: pgmap v567: 129 pgs: 129 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 700 KiB/s wr, 487 op/s 2024-09-05T02:09:40.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:40 smithi139 ceph-mon[34262]: pgmap v567: 129 pgs: 129 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 700 KiB/s wr, 487 op/s 2024-09-05T02:09:40.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:40 smithi103 ceph-mon[30902]: pgmap v567: 129 pgs: 129 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 700 KiB/s wr, 487 op/s 2024-09-05T02:09:40.745 DEBUG:teuthology.parallel:result is None 2024-09-05T02:09:40.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-05T02:09:40.795 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-05T02:09:40.795 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-05T02:09:40.806 INFO:tasks.quiescer:joining Quiescers 2024-09-05T02:09:40.806 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-05T02:09:40.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:09:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.664+0000 7fc8545a7640 1 -- 172.21.15.103:0/549513449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102570 msgr2=0x7fc84c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.664+0000 7fc8545a7640 1 --2- 172.21.15.103:0/549513449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102570 0x7fc84c1029b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009900 tx=0x7fc83c02a480 comp rx=0 tx=0).stop 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.664+0000 7fc8545a7640 1 -- 172.21.15.103:0/549513449 shutdown_connections 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.664+0000 7fc8545a7640 1 --2- 172.21.15.103:0/549513449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c102ef0 0x7fc84c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.664+0000 7fc8545a7640 1 --2- 172.21.15.103:0/549513449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102570 0x7fc84c1029b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.664+0000 7fc8545a7640 1 --2- 172.21.15.103:0/549513449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc84c100e20 0x7fc84c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.664+0000 7fc8545a7640 1 -- 172.21.15.103:0/549513449 >> 172.21.15.103:0/549513449 conn(0x7fc84c0fc7d0 msgr2=0x7fc84c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.664+0000 7fc8545a7640 1 -- 172.21.15.103:0/549513449 shutdown_connections 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.664+0000 7fc8545a7640 1 -- 172.21.15.103:0/549513449 wait complete. 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc8545a7640 1 Processor -- start 2024-09-05T02:09:41.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc8545a7640 1 -- start start 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc8545a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc84c100e20 0x7fc84c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc8545a7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c102570 0x7fc84c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc8545a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102ef0 0x7fc84c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc8545a7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c110ed0 con 0x7fc84c102ef0 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc8545a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c110d50 con 0x7fc84c100e20 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc8545a7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c111050 con 0x7fc84c102570 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852b44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102ef0 0x7fc84c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852b44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102ef0 0x7fc84c19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51516/0 (socket says 172.21.15.103:51516) 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852b44640 1 -- 172.21.15.103:0/318937898 learned_addr learned my addr 172.21.15.103:0/318937898 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc851b42640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c102570 0x7fc84c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852343640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc84c100e20 0x7fc84c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852b44640 1 -- 172.21.15.103:0/318937898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c102570 msgr2=0x7fc84c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:41.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852b44640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c102570 0x7fc84c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852b44640 1 -- 172.21.15.103:0/318937898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc84c100e20 msgr2=0x7fc84c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852b44640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc84c100e20 0x7fc84c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852b44640 1 -- 172.21.15.103:0/318937898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c19d4b0 con 0x7fc84c102ef0 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc851b42640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c102570 0x7fc84c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc852b44640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102ef0 0x7fc84c19cdd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc84400b970 tx=0x7fc84400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.665+0000 7fc8437fe640 1 -- 172.21.15.103:0/318937898 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844019070 con 0x7fc84c102ef0 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.666+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84c19d7a0 con 0x7fc84c102ef0 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.666+0000 7fc8437fe640 1 -- 172.21.15.103:0/318937898 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc844014440 con 0x7fc84c102ef0 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.666+0000 7fc8437fe640 1 -- 172.21.15.103:0/318937898 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440075d0 con 0x7fc84c102ef0 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.666+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84c1a5080 con 0x7fc84c102ef0 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.667+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84c101200 con 0x7fc84c102ef0 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.668+0000 7fc8437fe640 1 -- 172.21.15.103:0/318937898 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc84401c070 con 0x7fc84c102ef0 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.668+0000 7fc8437fe640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8380a2b80 0x7fc8380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:41.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.668+0000 7fc852343640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8380a2b80 0x7fc8380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:41.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.668+0000 7fc8437fe640 1 -- 172.21.15.103:0/318937898 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc8440ce9f0 con 0x7fc84c102ef0 2024-09-05T02:09:41.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.669+0000 7fc852343640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8380a2b80 0x7fc8380a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc84800a810 tx=0x7fc848008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:41.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.673+0000 7fc8437fe640 1 -- 172.21.15.103:0/318937898 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc84409b5a0 con 0x7fc84c102ef0 2024-09-05T02:09:41.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.875+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc84c10a0a0 con 0x7fc84c102ef0 2024-09-05T02:09:41.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.876+0000 7fc8437fe640 1 -- 172.21.15.103:0/318937898 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+12033 (secure 0 0 0) 0x7fc84409aea0 con 0x7fc84c102ef0 2024-09-05T02:09:41.878 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:09:41.879 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":42,"btime":"2024-09-05T02:09:29:302685+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:09:28.699103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:09:41.879 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 42 2024-09-05T02:09:41.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.878+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8380a2b80 msgr2=0x7fc8380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.878+0000 7fc8545a7640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8380a2b80 0x7fc8380a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc84800a810 tx=0x7fc848008040 comp rx=0 tx=0).stop 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.878+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102ef0 msgr2=0x7fc84c19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.878+0000 7fc8545a7640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102ef0 0x7fc84c19cdd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc84400b970 tx=0x7fc84400be30 comp rx=0 tx=0).stop 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.879+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 shutdown_connections 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.879+0000 7fc8545a7640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc84c102ef0 0x7fc84c19cdd0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.879+0000 7fc8545a7640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fc8380a2b80 0x7fc8380a5040 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.879+0000 7fc8545a7640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c102570 0x7fc84c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.879+0000 7fc8545a7640 1 --2- 172.21.15.103:0/318937898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc84c100e20 0x7fc84c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.879+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 >> 172.21.15.103:0/318937898 conn(0x7fc84c0fc7d0 msgr2=0x7fc84c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.879+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 shutdown_connections 2024-09-05T02:09:41.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:41.879+0000 7fc8545a7640 1 -- 172.21.15.103:0/318937898 wait complete. 2024-09-05T02:09:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:42 smithi167 ceph-mon[35122]: pgmap v568: 129 pgs: 129 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 940 KiB/s wr, 486 op/s 2024-09-05T02:09:42.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:42 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/318937898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:09:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:42 smithi139 ceph-mon[34262]: pgmap v568: 129 pgs: 129 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 940 KiB/s wr, 486 op/s 2024-09-05T02:09:42.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:42 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/318937898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:09:42.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:42 smithi103 ceph-mon[30902]: pgmap v568: 129 pgs: 129 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 940 KiB/s wr, 486 op/s 2024-09-05T02:09:42.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:42 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/318937898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:09:43.562 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --cancel --all' 2024-09-05T02:09:43.563 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --cancel --all 2024-09-05T02:09:43.820 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:09:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 -- 172.21.15.103:0/624217337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc102570 msgr2=0x7fd6bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/624217337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc102570 0x7fd6bc1029b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009900 tx=0x7fd6b002a480 comp rx=0 tx=0).stop 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 -- 172.21.15.103:0/624217337 shutdown_connections 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/624217337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6bc102ef0 0x7fd6bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/624217337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc102570 0x7fd6bc1029b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/624217337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6bc100e20 0x7fd6bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 -- 172.21.15.103:0/624217337 >> 172.21.15.103:0/624217337 conn(0x7fd6bc0fc7d0 msgr2=0x7fd6bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 -- 172.21.15.103:0/624217337 shutdown_connections 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 -- 172.21.15.103:0/624217337 wait complete. 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.331+0000 7fd6c08c4640 1 Processor -- start 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6c08c4640 1 -- start start 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6c08c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc100e20 0x7fd6bc199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6c08c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6bc102570 0x7fd6bc199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6c08c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6bc102ef0 0x7fd6bc1a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6c08c4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc110e70 con 0x7fd6bc100e20 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6c08c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc110cf0 con 0x7fd6bc102570 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6c08c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc110ff0 con 0x7fd6bc102ef0 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc100e20 0x7fd6bc199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc100e20 0x7fd6bc199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51526/0 (socket says 172.21.15.103:51526) 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6ba59c640 1 -- 172.21.15.103:0/3290803933 learned_addr learned my addr 172.21.15.103:0/3290803933 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6bc102570 0x7fd6bc199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6bad9d640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6bc102ef0 0x7fd6bc1a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6ba59c640 1 -- 172.21.15.103:0/3290803933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6bc102ef0 msgr2=0x7fd6bc1a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6bc102ef0 0x7fd6bc1a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6ba59c640 1 -- 172.21.15.103:0/3290803933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6bc102570 msgr2=0x7fd6bc199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6bc102570 0x7fd6bc199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6ba59c640 1 -- 172.21.15.103:0/3290803933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc1a48c0 con 0x7fd6bc100e20 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.332+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc100e20 0x7fd6bc199300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00b4a0 tx=0x7fd6ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.333+0000 7fd6ab7fe640 1 -- 172.21.15.103:0/3290803933 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac014070 con 0x7fd6bc100e20 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.333+0000 7fd6ab7fe640 1 -- 172.21.15.103:0/3290803933 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6ac010ac0 con 0x7fd6bc100e20 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.333+0000 7fd6c08c4640 1 -- 172.21.15.103:0/3290803933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc1a4a90 con 0x7fd6bc100e20 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.333+0000 7fd6ab7fe640 1 -- 172.21.15.103:0/3290803933 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac017c30 con 0x7fd6bc100e20 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.333+0000 7fd6c08c4640 1 -- 172.21.15.103:0/3290803933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc1a4fa0 con 0x7fd6bc100e20 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.335+0000 7fd6ab7fe640 1 -- 172.21.15.103:0/3290803933 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd6ac017070 con 0x7fd6bc100e20 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.335+0000 7fd6c2c9d740 1 Processor -- start 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.335+0000 7fd6c2c9d740 1 -- start start 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34eefa00 0x55dd34eefde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34ef0320 0x55dd34efa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34efabe0 0x55dd34efcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dd34e54200 con 0x55dd34ef0320 2024-09-05T02:09:44.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dd34dfadc0 con 0x55dd34efabe0 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dd34e501c0 con 0x55dd34eefa00 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34ef0320 0x55dd34efa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34ef0320 0x55dd34efa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51528/0 (socket says 172.21.15.103:51528) 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6ba59c640 1 -- 172.21.15.103:0/3695934787 learned_addr learned my addr 172.21.15.103:0/3695934787 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6ab7fe640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6a40a2c00 0x7fd6a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6bad9d640 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34eefa00 0x55dd34eefde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34efabe0 0x55dd34efcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6bad9d640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6a40a2c00 0x7fd6a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6bad9d640 1 -- 172.21.15.103:0/3695934787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34efabe0 msgr2=0x55dd34efcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6bad9d640 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34efabe0 0x55dd34efcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6bad9d640 1 -- 172.21.15.103:0/3695934787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34ef0320 msgr2=0x55dd34efa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:44.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6ab7fe640 1 -- 172.21.15.103:0/3290803933 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd6ac0ce980 con 0x7fd6bc100e20 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6bad9d640 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34ef0320 0x55dd34efa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6bad9d640 1 -- 172.21.15.103:0/3695934787 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd34bc5620 con 0x55dd34eefa00 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.336+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34ef0320 0x55dd34efa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.337+0000 7fd6bad9d640 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34eefa00 0x55dd34eefde0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4005fd0 tx=0x7fd6b4014ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b279ff7b8502d076 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.337+0000 7fd6bad9d640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6a40a2c00 0x7fd6a40a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd6b401c000 tx=0x7fd6b401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.337+0000 7fd6a8ff9640 1 -- 172.21.15.103:0/3695934787 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4023070 con 0x55dd34eefa00 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.337+0000 7fd6a8ff9640 1 -- 172.21.15.103:0/3695934787 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6b4012dd0 con 0x55dd34eefa00 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.337+0000 7fd6a8ff9640 1 -- 172.21.15.103:0/3695934787 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b401ed60 con 0x55dd34eefa00 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.337+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/3695934787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34eefa00 msgr2=0x55dd34eefde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.337+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34eefa00 0x55dd34eefde0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4005fd0 tx=0x7fd6b4014ed0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.337+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/3695934787 shutdown_connections 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.337+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34efabe0 0x55dd34efcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34ef0320 0x55dd34efa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/3695934787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34eefa00 0x55dd34eefde0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/3695934787 >> 172.21.15.103:0/3695934787 conn(0x55dd34eebf90 msgr2=0x55dd34eec370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/3695934787 shutdown_connections 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/3695934787 wait complete. 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 -- start start 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 Processor -- start 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34efa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34ef77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34efabe0 0x55dd34ef7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dd34efd7b0 con 0x55dd34eefa00 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dd34deec50 con 0x55dd34ef0320 2024-09-05T02:09:44.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dd34efd930 con 0x55dd34efabe0 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6c2c9d740 10 client.0 _ll_register_callbacks cb 0x55dd34e00ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.338+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34efa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34efa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51534/0 (socket says 172.21.15.103:51534) 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6ba59c640 1 -- 172.21.15.103:0/1069712985 learned_addr learned my addr 172.21.15.103:0/1069712985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34ef77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6bad9d640 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34efabe0 0x55dd34ef7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6ba59c640 1 -- 172.21.15.103:0/1069712985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34efabe0 msgr2=0x55dd34ef7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34efabe0 0x55dd34ef7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6ba59c640 1 -- 172.21.15.103:0/1069712985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 msgr2=0x55dd34ef77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34ef77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6ba59c640 1 -- 172.21.15.103:0/1069712985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd34ef8570 con 0x55dd34eefa00 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34ef77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:09:44.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34efa530 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0cc120 tx=0x7fd6ac017360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85acad7e00b3f515 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1069712985 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac01e070 con 0x55dd34eefa00 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1069712985 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd6ac097550 con 0x55dd34eefa00 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1069712985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6b4012dd0 con 0x55dd34eefa00 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1069712985 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd6ac09aee0 con 0x55dd34eefa00 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6c2c9d740 10 client.16254 fetch_fsmap learned FSMap version 42 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1069712985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dd34f093c0 con 0x55dd34eefa00 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.339+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1069712985 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6b4012dd0 con 0x55dd34eefa00 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1069712985 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 42) ==== 8450+0+0 (secure 0 0 0) 0x7fd6ac09b140 con 0x55dd34eefa00 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6c2c9d740 10 client.16254 fetch_fsmap finished waiting for FSMap version 42 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6c2c9d740 10 client.16254 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:09:44.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6c2c9d740 20 client.16254 populate_metadata read hostname 'smithi103' 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd34f0a880 0x55dd34f0ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6c2c9d740 4 client.16254 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1069712985 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd34e61770 con 0x55dd34f0a880 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd34f0a880 0x55dd34f0ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd34f0a880 0x55dd34f0ccf0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8f36324a2450b4 server_cookie=76aae45840410d0d in_seq=0 out_seq=0 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.340+0000 7fd6a1ffb640 10 client.16254 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1069712985 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd34e61770 con 0x55dd34f0a880 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6a1ffb640 10 client.16254 handle_command_reply: tid=0 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 client.16254 shutdown 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1069712985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 msgr2=0x55dd34efa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34efa530 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0cc120 tx=0x7fd6ac017360 comp rx=0 tx=0).stop 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1069712985 shutdown_connections 2024-09-05T02:09:44.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd34f0a880 0x55dd34f0ccf0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34efabe0 0x55dd34ef7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34ef77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6a1ffb640 0 client.16254 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1069712985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34efa530 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1069712985 >> 172.21.15.103:0/1069712985 conn(0x55dd34eebf90 msgr2=0x55dd34eeb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1069712985 shutdown_connections 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.341+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1069712985 wait complete. 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.342+0000 7fd6c2c9d740 20 client.16254 trim_cache size 0 max 16384 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6c2c9d740 1 Processor -- start 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6c2c9d740 1 -- start start 2024-09-05T02:09:44.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f06a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34eeb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34efabe0 0x55dd34f125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc1aed90 con 0x55dd34eefa00 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc1b0bd0 con 0x55dd34ef0320 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc1af520 con 0x55dd34efabe0 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f06a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f06a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51548/0 (socket says 172.21.15.103:51548) 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.358+0000 7fd6ba59c640 1 -- 172.21.15.103:0/2000446357 learned_addr learned my addr 172.21.15.103:0/2000446357 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/2000446357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34eeb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6ba59c640 1 -- 172.21.15.103:0/2000446357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34efabe0 msgr2=0x55dd34f125c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/2000446357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34efabe0 0x55dd34f125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6ba59c640 1 -- 172.21.15.103:0/2000446357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 msgr2=0x55dd34eeb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/2000446357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34eeb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6ba59c640 1 -- 172.21.15.103:0/2000446357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd34f0d370 con 0x55dd34eefa00 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/2000446357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34eeb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/2000446357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f06a10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000c00 tx=0x7fd6ac00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40469ee6b7bae91 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6a3fff640 1 -- 172.21.15.103:0/2000446357 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac01e070 con 0x55dd34eefa00 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6a3fff640 1 -- 172.21.15.103:0/2000446357 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6ac09f1e0 con 0x55dd34eefa00 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.359+0000 7fd6a3fff640 1 -- 172.21.15.103:0/2000446357 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0a3610 con 0x55dd34eefa00 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/2000446357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 msgr2=0x55dd34f06a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/2000446357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f06a10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000c00 tx=0x7fd6ac00bf90 comp rx=0 tx=0).stop 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/2000446357 shutdown_connections 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/2000446357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34efabe0 0x55dd34f125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/2000446357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34ef0320 0x55dd34eeb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/2000446357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f06a10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/2000446357 >> 172.21.15.103:0/2000446357 conn(0x55dd34eebf90 msgr2=0x55dd34eea9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/2000446357 shutdown_connections 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/2000446357 wait complete. 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 -- start start 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 Processor -- start 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34ef0320 0x55dd34efe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34efabe0 0x55dd34efecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc1b0440 con 0x55dd34eefa00 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc1afcb0 con 0x55dd34efabe0 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc1acd70 con 0x55dd34ef0320 2024-09-05T02:09:44.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6c2c9d740 10 client.0 _ll_register_callbacks cb 0x7fd6ac0175f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f033b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51558/0 (socket says 172.21.15.103:51558) 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.360+0000 7fd6ba59c640 1 -- 172.21.15.103:0/1804883144 learned_addr learned my addr 172.21.15.103:0/1804883144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6ba59c640 1 -- 172.21.15.103:0/1804883144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34ef0320 msgr2=0x55dd34efe6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34ef0320 0x55dd34efe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6ba59c640 1 -- 172.21.15.103:0/1804883144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34efabe0 msgr2=0x55dd34efecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34efabe0 0x55dd34efecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6ba59c640 1 -- 172.21.15.103:0/1804883144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd34eff500 con 0x55dd34eefa00 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6ba59c640 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f033b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0176a0 tx=0x7fd6ac09aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506d701f4007363 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1804883144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac01e070 con 0x55dd34eefa00 2024-09-05T02:09:44.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1804883144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd6ac010610 con 0x55dd34eefa00 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1804883144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6ac09f1e0 con 0x55dd34eefa00 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1804883144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd6ac010890 con 0x55dd34eefa00 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6c2c9d740 10 client.16260 fetch_fsmap learned FSMap version 42 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1804883144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dd34eff500 con 0x55dd34eefa00 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.361+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1804883144 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6ac09f1e0 con 0x55dd34eefa00 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1804883144 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 42) ==== 8450+0+0 (secure 0 0 0) 0x7fd6ac099f10 con 0x55dd34eefa00 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6c2c9d740 10 client.16260 fetch_fsmap finished waiting for FSMap version 42 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6c2c9d740 10 client.16260 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6c2c9d740 20 client.16260 populate_metadata read hostname 'smithi103' 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd34f75830 0x55dd34f77c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6c2c9d740 4 client.16260 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-05T02:09:44.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1804883144 --> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55dd34e5bc80 con 0x55dd34f75830 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd34f75830 0x55dd34f77c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6b9d9b640 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd34f75830 0x55dd34f77c20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d410e3b9f16def2 server_cookie=5f0028f60b0362cd in_seq=0 out_seq=0 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.362+0000 7fd6a1ffb640 10 client.16260 ms_handle_connect on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6a1ffb640 1 -- 172.21.15.103:0/1804883144 <== mds.0 v2:172.21.15.139:6838/3082363979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55dd34e5bc80 con 0x55dd34f75830 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6a1ffb640 10 client.16260 handle_command_reply: tid=0 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 client.16260 shutdown 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1804883144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 msgr2=0x55dd34f033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f033b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0176a0 tx=0x7fd6ac09aa20 comp rx=0 tx=0).stop 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1804883144 shutdown_connections 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.139:6838/3082363979,v1:172.21.15.139:6839/3082363979] conn(0x55dd34f75830 0x55dd34f77c20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dd34efabe0 0x55dd34efecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd34ef0320 0x55dd34efe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6a1ffb640 0 client.16260 ms_handle_reset on v2:172.21.15.139:6838/3082363979 2024-09-05T02:09:44.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 --2- 172.21.15.103:0/1804883144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd34eefa00 0x55dd34f033b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1804883144 >> 172.21.15.103:0/1804883144 conn(0x55dd34eebf90 msgr2=0x55dd34ef1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:44.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1804883144 shutdown_connections 2024-09-05T02:09:44.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 1 -- 172.21.15.103:0/1804883144 wait complete. 2024-09-05T02:09:44.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.363+0000 7fd6c2c9d740 20 client.16260 trim_cache size 0 max 16384 2024-09-05T02:09:44.382 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:09:44.383 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 42, 2024-09-05T02:09:44.383 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24497, 2024-09-05T02:09:44.383 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 20, 2024-09-05T02:09:44.383 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": {} 2024-09-05T02:09:44.383 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 -- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6a40a2c00 msgr2=0x7fd6a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6a40a2c00 0x7fd6a40a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd6b401c000 tx=0x7fd6b401e000 comp rx=0 tx=0).stop 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 -- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc100e20 msgr2=0x7fd6bc199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc100e20 0x7fd6bc199300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00b4a0 tx=0x7fd6ac00b960 comp rx=0 tx=0).stop 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 -- 172.21.15.103:0/3290803933 shutdown_connections 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd6a40a2c00 0x7fd6a40a50c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6bc102ef0 0x7fd6bc1a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6bc102570 0x7fd6bc199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 --2- 172.21.15.103:0/3290803933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6bc100e20 0x7fd6bc199300 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 -- 172.21.15.103:0/3290803933 >> 172.21.15.103:0/3290803933 conn(0x7fd6bc0fc7d0 msgr2=0x7fd6bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 -- 172.21.15.103:0/3290803933 shutdown_connections 2024-09-05T02:09:44.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:44.364+0000 7fd6c08c4640 1 -- 172.21.15.103:0/3290803933 wait complete. 2024-09-05T02:09:44.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:44 smithi167 ceph-mon[35122]: pgmap v569: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 894 KiB/s wr, 420 op/s 2024-09-05T02:09:44.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:44 smithi139 ceph-mon[34262]: pgmap v569: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 894 KiB/s wr, 420 op/s 2024-09-05T02:09:44.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:44 smithi103 ceph-mon[30902]: pgmap v569: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 894 KiB/s wr, 420 op/s 2024-09-05T02:09:45.182 INFO:tasks.quiescer:done joining Quiescers 2024-09-05T02:09:45.182 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-05T02:09:45.191 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi103.front.sepia.ceph.com 2024-09-05T02:09:45.200 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-05T02:09:45.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T02:09:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 -- 172.21.15.103:0/1464949753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 msgr2=0x7f0a44101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/1464949753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 0x7f0a44101350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009900 tx=0x7f0a3c02a480 comp rx=0 tx=0).stop 2024-09-05T02:09:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 -- 172.21.15.103:0/1464949753 shutdown_connections 2024-09-05T02:09:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/1464949753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a44102620 0x7f0a44108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/1464949753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a44101c60 0x7f0a441020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/1464949753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 0x7f0a44101350 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 -- 172.21.15.103:0/1464949753 >> 172.21.15.103:0/1464949753 conn(0x7f0a440fb950 msgr2=0x7f0a440fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 -- 172.21.15.103:0/1464949753 shutdown_connections 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 -- 172.21.15.103:0/1464949753 wait complete. 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 Processor -- start 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.346+0000 7f0a4af4b640 1 -- start start 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a4af4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 0x7f0a44100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a4af4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a44101c60 0x7f0a4419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a4af4b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a44102620 0x7f0a4419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a4af4b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4410c580 con 0x7f0a44100f50 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a4af4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4410c400 con 0x7f0a44101c60 2024-09-05T02:09:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a4af4b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4410c700 con 0x7f0a44102620 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a48ce7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 0x7f0a44100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a48ce7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 0x7f0a44100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51580/0 (socket says 172.21.15.103:51580) 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a48ce7640 1 -- 172.21.15.103:0/2695409692 learned_addr learned my addr 172.21.15.103:0/2695409692 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a48ce7640 1 -- 172.21.15.103:0/2695409692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a44102620 msgr2=0x7f0a4419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a48ce7640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a44102620 0x7f0a4419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a48ce7640 1 -- 172.21.15.103:0/2695409692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a44101c60 msgr2=0x7f0a4419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a48ce7640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a44101c60 0x7f0a4419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a48ce7640 1 -- 172.21.15.103:0/2695409692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4419fa50 con 0x7f0a44100f50 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a48ce7640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 0x7f0a44100610 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0314e0 tx=0x7f0a3c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a41ffb640 1 -- 172.21.15.103:0/2695409692 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c03f030 con 0x7f0a44100f50 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a41ffb640 1 -- 172.21.15.103:0/2695409692 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a3c003cb0 con 0x7f0a44100f50 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4419fc80 con 0x7f0a44100f50 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.347+0000 7f0a41ffb640 1 -- 172.21.15.103:0/2695409692 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c035070 con 0x7f0a44100f50 2024-09-05T02:09:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.348+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a441a01c0 con 0x7f0a44100f50 2024-09-05T02:09:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.348+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a44107f50 con 0x7f0a44100f50 2024-09-05T02:09:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.350+0000 7f0a41ffb640 1 -- 172.21.15.103:0/2695409692 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0a3c042030 con 0x7f0a44100f50 2024-09-05T02:09:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.350+0000 7f0a41ffb640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0a300a2c10 0x7f0a300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.350+0000 7f0a43fff640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0a300a2c10 0x7f0a300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.350+0000 7f0a41ffb640 1 -- 172.21.15.103:0/2695409692 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0a3c0efa30 con 0x7f0a44100f50 2024-09-05T02:09:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.351+0000 7f0a43fff640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0a300a2c10 0x7f0a300a50d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0a44100380 tx=0x7f0a34005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:45.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.354+0000 7f0a41ffb640 1 -- 172.21.15.103:0/2695409692 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a3c0bc5e0 con 0x7f0a44100f50 2024-09-05T02:09:45.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.525+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a44062e90 con 0x7f0a44100f50 2024-09-05T02:09:45.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.527+0000 7f0a41ffb640 1 -- 172.21.15.103:0/2695409692 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24926 (secure 0 0 0) 0x7f0a3c0bbee0 con 0x7f0a44100f50 2024-09-05T02:09:45.529 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:09:45.529 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":90,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:02:27.192641+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:09:45.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0a300a2c10 msgr2=0x7f0a300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0a300a2c10 0x7f0a300a50d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0a44100380 tx=0x7f0a34005f70 comp rx=0 tx=0).stop 2024-09-05T02:09:45.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 msgr2=0x7f0a44100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 0x7f0a44100610 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0314e0 tx=0x7f0a3c0042e0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 shutdown_connections 2024-09-05T02:09:45.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f0a300a2c10 0x7f0a300a50d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a44102620 0x7f0a4419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a44101c60 0x7f0a4419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 --2- 172.21.15.103:0/2695409692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a44100f50 0x7f0a44100610 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 >> 172.21.15.103:0/2695409692 conn(0x7f0a440fb950 msgr2=0x7f0a44105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:45.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 shutdown_connections 2024-09-05T02:09:45.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.529+0000 7f0a4af4b640 1 -- 172.21.15.103:0/2695409692 wait complete. 2024-09-05T02:09:45.542 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T01:55:35.312049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-05T02:00:11.283498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-05T02:00:13.180482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-05T02:00:19.622395+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-05T02:09:45.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-05T02:09:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.643+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/2729217242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 msgr2=0x7f4d24101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.643+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/2729217242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 0x7f4d24101f30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009900 tx=0x7f4d2002a410 comp rx=0 tx=0).stop 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/2729217242 shutdown_connections 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/2729217242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d241036e0 0x7f4d24109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/2729217242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d24102d20 0x7f4d241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/2729217242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 0x7f4d24101f30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/2729217242 >> 172.21.15.103:0/2729217242 conn(0x7f4d240fd2b0 msgr2=0x7f4d240ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/2729217242 shutdown_connections 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/2729217242 wait complete. 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 Processor -- start 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 -- start start 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 0x7f4d24075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d24102d20 0x7f4d240745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241036e0 0x7f4d24074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2410d660 con 0x7f4d24101b30 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2410d4e0 con 0x7f4d241036e0 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d2bdcf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2410d7e0 con 0x7f4d24102d20 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.644+0000 7f4d29b6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 0x7f4d24075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d29b6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 0x7f4d24075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51596/0 (socket says 172.21.15.103:51596) 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d29b6b640 1 -- 172.21.15.103:0/4135187543 learned_addr learned my addr 172.21.15.103:0/4135187543 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d2936a640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d24102d20 0x7f4d240745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d2a36c640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241036e0 0x7f4d24074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d29b6b640 1 -- 172.21.15.103:0/4135187543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d24102d20 msgr2=0x7f4d240745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d29b6b640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d24102d20 0x7f4d240745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d29b6b640 1 -- 172.21.15.103:0/4135187543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241036e0 msgr2=0x7f4d24074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d29b6b640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241036e0 0x7f4d24074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d29b6b640 1 -- 172.21.15.103:0/4135187543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d240753b0 con 0x7f4d24101b30 2024-09-05T02:09:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d2a36c640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241036e0 0x7f4d24074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:09:45.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d29b6b640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 0x7f4d24075fa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4d200314e0 tx=0x7f4d200042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:45.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d1affd640 1 -- 172.21.15.103:0/4135187543 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2003f030 con 0x7f4d24101b30 2024-09-05T02:09:45.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d1affd640 1 -- 172.21.15.103:0/4135187543 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d20003cb0 con 0x7f4d24101b30 2024-09-05T02:09:45.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d1affd640 1 -- 172.21.15.103:0/4135187543 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20035070 con 0x7f4d24101b30 2024-09-05T02:09:45.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d241a1e60 con 0x7f4d24101b30 2024-09-05T02:09:45.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.645+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d241a22c0 con 0x7f4d24101b30 2024-09-05T02:09:45.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.646+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d24109010 con 0x7f4d24101b30 2024-09-05T02:09:45.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.647+0000 7f4d1affd640 1 -- 172.21.15.103:0/4135187543 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4d20042030 con 0x7f4d24101b30 2024-09-05T02:09:45.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.648+0000 7f4d1affd640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d100a2c10 0x7f4d100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.648+0000 7f4d2936a640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d100a2c10 0x7f4d100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:45.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.648+0000 7f4d1affd640 1 -- 172.21.15.103:0/4135187543 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4d200efde0 con 0x7f4d24101b30 2024-09-05T02:09:45.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.648+0000 7f4d2936a640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d100a2c10 0x7f4d100a50d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4d24075d10 tx=0x7f4d1400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:45.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.651+0000 7f4d1affd640 1 -- 172.21.15.103:0/4135187543 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d200bca90 con 0x7f4d24101b30 2024-09-05T02:09:45.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.817+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d2402da30 con 0x7f4d24101b30 2024-09-05T02:09:45.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.818+0000 7f4d1affd640 1 -- 172.21.15.103:0/4135187543 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f4d200bc390 con 0x7f4d24101b30 2024-09-05T02:09:45.820 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-05T02:09:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d100a2c10 msgr2=0x7f4d100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d100a2c10 0x7f4d100a50d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4d24075d10 tx=0x7f4d1400a4c0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 msgr2=0x7f4d24075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 0x7f4d24075fa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4d200314e0 tx=0x7f4d200042e0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 shutdown_connections 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241036e0 0x7f4d24074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4d100a2c10 0x7f4d100a50d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d24102d20 0x7f4d240745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 --2- 172.21.15.103:0/4135187543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d24101b30 0x7f4d24075fa0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 >> 172.21.15.103:0/4135187543 conn(0x7f4d240fd2b0 msgr2=0x7f4d241061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 shutdown_connections 2024-09-05T02:09:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.820+0000 7f4d2bdcf640 1 -- 172.21.15.103:0/4135187543 wait complete. 2024-09-05T02:09:45.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-05T02:09:45.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.934+0000 7f4bf7053640 1 -- 172.21.15.103:0/541547142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf01025b0 msgr2=0x7f4bf01049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.934+0000 7f4bf7053640 1 --2- 172.21.15.103:0/541547142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf01025b0 0x7f4bf01049a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009900 tx=0x7f4bec02a410 comp rx=0 tx=0).stop 2024-09-05T02:09:45.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 -- 172.21.15.103:0/541547142 shutdown_connections 2024-09-05T02:09:45.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 --2- 172.21.15.103:0/541547142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bf0107810 0x7f4bf0109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 --2- 172.21.15.103:0/541547142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf0104ee0 0x7f4bf01072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 --2- 172.21.15.103:0/541547142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf01025b0 0x7f4bf01049a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 -- 172.21.15.103:0/541547142 >> 172.21.15.103:0/541547142 conn(0x7f4bf00fc1b0 msgr2=0x7f4bf00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 -- 172.21.15.103:0/541547142 shutdown_connections 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 -- 172.21.15.103:0/541547142 wait complete. 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 Processor -- start 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 -- start start 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01025b0 0x7f4bf0197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0104ee0 0x7f4bf0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bf0107810 0x7f4bf019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf010d6c0 con 0x7f4bf0104ee0 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf010d540 con 0x7f4bf0107810 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4bf7053640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf010d840 con 0x7f4bf01025b0 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0104ee0 0x7f4bf0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.935+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0104ee0 0x7f4bf0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51618/0 (socket says 172.21.15.103:51618) 2024-09-05T02:09:45.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be7fff640 1 -- 172.21.15.103:0/292562234 learned_addr learned my addr 172.21.15.103:0/292562234 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be7fff640 1 -- 172.21.15.103:0/292562234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01025b0 msgr2=0x7f4bf0197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be7fff640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01025b0 0x7f4bf0197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be7fff640 1 -- 172.21.15.103:0/292562234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bf0107810 msgr2=0x7f4bf019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4bf55f0640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bf0107810 0x7f4bf019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be7fff640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bf0107810 0x7f4bf019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be7fff640 1 -- 172.21.15.103:0/292562234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf019fb10 con 0x7f4bf0104ee0 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4bf55f0640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bf0107810 0x7f4bf019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be7fff640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0104ee0 0x7f4bf0198430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c970 tx=0x7f4be000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be5ffb640 1 -- 172.21.15.103:0/292562234 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0007bb0 con 0x7f4bf0104ee0 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be5ffb640 1 -- 172.21.15.103:0/292562234 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4be0007d50 con 0x7f4bf0104ee0 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4be5ffb640 1 -- 172.21.15.103:0/292562234 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0018730 con 0x7f4bf0104ee0 2024-09-05T02:09:45.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf019fe00 con 0x7f4bf0104ee0 2024-09-05T02:09:45.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.936+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf01a02c0 con 0x7f4bf0104ee0 2024-09-05T02:09:45.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.937+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf00fdca0 con 0x7f4bf0104ee0 2024-09-05T02:09:45.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.938+0000 7f4be5ffb640 1 -- 172.21.15.103:0/292562234 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4be000f450 con 0x7f4bf0104ee0 2024-09-05T02:09:45.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.939+0000 7f4be5ffb640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bd80a2c10 0x7f4bd80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:45.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.939+0000 7f4bf4def640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bd80a2c10 0x7f4bd80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:45.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.939+0000 7f4be5ffb640 1 -- 172.21.15.103:0/292562234 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4be00cf460 con 0x7f4bf0104ee0 2024-09-05T02:09:45.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.940+0000 7f4bf4def640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bd80a2c10 0x7f4bd80a50d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0096c0 tx=0x7f4bec0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:45.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:45.943+0000 7f4be5ffb640 1 -- 172.21.15.103:0/292562234 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4be0014030 con 0x7f4bf0104ee0 2024-09-05T02:09:46.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.108+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4bf0062df0 con 0x7f4bf0104ee0 2024-09-05T02:09:46.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.109+0000 7f4be5ffb640 1 -- 172.21.15.103:0/292562234 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f4be009c010 con 0x7f4bf0104ee0 2024-09-05T02:09:46.111 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-05T02:09:46.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bd80a2c10 msgr2=0x7f4bd80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bd80a2c10 0x7f4bd80a50d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0096c0 tx=0x7f4bec0314a0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0104ee0 msgr2=0x7f4bf0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0104ee0 0x7f4bf0198430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c970 tx=0x7f4be000ce30 comp rx=0 tx=0).stop 2024-09-05T02:09:46.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 shutdown_connections 2024-09-05T02:09:46.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bf0107810 0x7f4bf019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0104ee0 0x7f4bf0198430 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4bd80a2c10 0x7f4bd80a50d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 --2- 172.21.15.103:0/292562234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01025b0 0x7f4bf0197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 >> 172.21.15.103:0/292562234 conn(0x7f4bf00fc1b0 msgr2=0x7f4bf0102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:46.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 shutdown_connections 2024-09-05T02:09:46.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.111+0000 7f4bf7053640 1 -- 172.21.15.103:0/292562234 wait complete. 2024-09-05T02:09:46.124 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-05T02:09:46.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.225+0000 7faf13e27640 1 -- 172.21.15.103:0/1547587535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 msgr2=0x7faf0c109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.225+0000 7faf13e27640 1 --2- 172.21.15.103:0/1547587535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 0x7faf0c109c60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7faf04009900 tx=0x7faf0402a3f0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.225+0000 7faf13e27640 1 -- 172.21.15.103:0/1547587535 shutdown_connections 2024-09-05T02:09:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.225+0000 7faf13e27640 1 --2- 172.21.15.103:0/1547587535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 0x7faf0c109c60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.225+0000 7faf13e27640 1 --2- 172.21.15.103:0/1547587535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0c104ec0 0x7faf0c1072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.225+0000 7faf13e27640 1 --2- 172.21.15.103:0/1547587535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf0c102590 0x7faf0c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.225+0000 7faf13e27640 1 -- 172.21.15.103:0/1547587535 >> 172.21.15.103:0/1547587535 conn(0x7faf0c0fc1b0 msgr2=0x7faf0c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.225+0000 7faf13e27640 1 -- 172.21.15.103:0/1547587535 shutdown_connections 2024-09-05T02:09:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.225+0000 7faf13e27640 1 -- 172.21.15.103:0/1547587535 wait complete. 2024-09-05T02:09:46.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf13e27640 1 Processor -- start 2024-09-05T02:09:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf13e27640 1 -- start start 2024-09-05T02:09:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf13e27640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0c102590 0x7faf0c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf13e27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf0c104ec0 0x7faf0c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf13e27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 0x7faf0c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf13e27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c10d680 con 0x7faf0c1077f0 2024-09-05T02:09:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf13e27640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c10d500 con 0x7faf0c104ec0 2024-09-05T02:09:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf13e27640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c10d800 con 0x7faf0c102590 2024-09-05T02:09:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf123c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 0x7faf0c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf123c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 0x7faf0c19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51632/0 (socket says 172.21.15.103:51632) 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf123c4640 1 -- 172.21.15.103:0/3071369242 learned_addr learned my addr 172.21.15.103:0/3071369242 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf11bc3640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0c102590 0x7faf0c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf113c2640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf0c104ec0 0x7faf0c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf123c4640 1 -- 172.21.15.103:0/3071369242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0c102590 msgr2=0x7faf0c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf123c4640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0c102590 0x7faf0c197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf123c4640 1 -- 172.21.15.103:0/3071369242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf0c104ec0 msgr2=0x7faf0c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf123c4640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf0c104ec0 0x7faf0c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf123c4640 1 -- 172.21.15.103:0/3071369242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0c19f960 con 0x7faf0c1077f0 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.226+0000 7faf123c4640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 0x7faf0c19f2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7faf04005940 tx=0x7faf04004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.227+0000 7faf0affd640 1 -- 172.21.15.103:0/3071369242 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0403f030 con 0x7faf0c1077f0 2024-09-05T02:09:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.227+0000 7faf0affd640 1 -- 172.21.15.103:0/3071369242 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf04003cb0 con 0x7faf0c1077f0 2024-09-05T02:09:46.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.227+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0c19fbf0 con 0x7faf0c1077f0 2024-09-05T02:09:46.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.227+0000 7faf0affd640 1 -- 172.21.15.103:0/3071369242 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04035070 con 0x7faf0c1077f0 2024-09-05T02:09:46.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.227+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0c1a00e0 con 0x7faf0c1077f0 2024-09-05T02:09:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.228+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0c0fdca0 con 0x7faf0c1077f0 2024-09-05T02:09:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.229+0000 7faf0affd640 1 -- 172.21.15.103:0/3071369242 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7faf04042030 con 0x7faf0c1077f0 2024-09-05T02:09:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.229+0000 7faf0affd640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faef40a2b90 0x7faef40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.230+0000 7faf11bc3640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faef40a2b90 0x7faef40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.230+0000 7faf0affd640 1 -- 172.21.15.103:0/3071369242 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7faf040eff10 con 0x7faf0c1077f0 2024-09-05T02:09:46.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.230+0000 7faf11bc3640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faef40a2b90 0x7faef40a5050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faefc007350 tx=0x7faefc0072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:46.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.233+0000 7faf0affd640 1 -- 172.21.15.103:0/3071369242 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf040bcac0 con 0x7faf0c1077f0 2024-09-05T02:09:46.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:46 smithi103 ceph-mon[30902]: pgmap v570: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 737 KiB/s wr, 432 op/s 2024-09-05T02:09:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:46 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2695409692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:09:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:46 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/4135187543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T02:09:46.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:46 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/292562234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T02:09:46.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.422+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faf0c062e90 con 0x7faf0c1077f0 2024-09-05T02:09:46.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.422+0000 7faf0affd640 1 -- 172.21.15.103:0/3071369242 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7faf040bc3c0 con 0x7faf0c1077f0 2024-09-05T02:09:46.424 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-09-05T02:09:46.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.424+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faef40a2b90 msgr2=0x7faef40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.424+0000 7faf13e27640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faef40a2b90 0x7faef40a5050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faefc007350 tx=0x7faefc0072e0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.424+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 msgr2=0x7faf0c19f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.424+0000 7faf13e27640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 0x7faf0c19f2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7faf04005940 tx=0x7faf04004340 comp rx=0 tx=0).stop 2024-09-05T02:09:46.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.425+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 shutdown_connections 2024-09-05T02:09:46.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.425+0000 7faf13e27640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf0c1077f0 0x7faf0c19f2f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.425+0000 7faf13e27640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf0c104ec0 0x7faf0c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.425+0000 7faf13e27640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7faef40a2b90 0x7faef40a5050 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.425+0000 7faf13e27640 1 --2- 172.21.15.103:0/3071369242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0c102590 0x7faf0c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.425+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 >> 172.21.15.103:0/3071369242 conn(0x7faf0c0fc1b0 msgr2=0x7faf0c102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:46.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.425+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 shutdown_connections 2024-09-05T02:09:46.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.425+0000 7faf13e27640 1 -- 172.21.15.103:0/3071369242 wait complete. 2024-09-05T02:09:46.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-05T02:09:46.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.537+0000 7f960e7d6640 1 -- 172.21.15.103:0/3133968581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96081036c0 msgr2=0x7f9608109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.537+0000 7f960e7d6640 1 --2- 172.21.15.103:0/3133968581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96081036c0 0x7f9608109f50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9600009ca0 tx=0x7f960002a8c0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 -- 172.21.15.103:0/3133968581 shutdown_connections 2024-09-05T02:09:46.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 --2- 172.21.15.103:0/3133968581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96081036c0 0x7f9608109f50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 --2- 172.21.15.103:0/3133968581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9608102d00 0x7f9608103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 --2- 172.21.15.103:0/3133968581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9608101b10 0x7f9608101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 -- 172.21.15.103:0/3133968581 >> 172.21.15.103:0/3133968581 conn(0x7f96080fd2b0 msgr2=0x7f96080ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:46.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 -- 172.21.15.103:0/3133968581 shutdown_connections 2024-09-05T02:09:46.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 -- 172.21.15.103:0/3133968581 wait complete. 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 Processor -- start 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 -- start start 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9608101b10 0x7f9608075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9608102d00 0x7f96080745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.538+0000 7f960e7d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96081036c0 0x7f9608074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f960e7d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f960810d640 con 0x7f9608101b10 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f960e7d6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f960810d4c0 con 0x7f96081036c0 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f960e7d6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f960810d7c0 con 0x7f9608102d00 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f9607fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9608101b10 0x7f9608075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f9607fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9608101b10 0x7f9608075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51636/0 (socket says 172.21.15.103:51636) 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f9607fff640 1 -- 172.21.15.103:0/4046729883 learned_addr learned my addr 172.21.15.103:0/4046729883 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f96077fe640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9608102d00 0x7f96080745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f9607fff640 1 -- 172.21.15.103:0/4046729883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9608102d00 msgr2=0x7f96080745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f9607fff640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9608102d00 0x7f96080745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f9607fff640 1 -- 172.21.15.103:0/4046729883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96081036c0 msgr2=0x7f9608074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f9607fff640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96081036c0 0x7f9608074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f9607fff640 1 -- 172.21.15.103:0/4046729883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96080753b0 con 0x7f9608101b10 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f96077fe640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9608102d00 0x7f96080745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f9607fff640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9608101b10 0x7f9608075fa0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f95f800d950 tx=0x7f95f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f96057fa640 1 -- 172.21.15.103:0/4046729883 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8017070 con 0x7f9608101b10 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f96057fa640 1 -- 172.21.15.103:0/4046729883 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95f8009c30 con 0x7f9608101b10 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96081a1e70 con 0x7f9608101b10 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f96057fa640 1 -- 172.21.15.103:0/4046729883 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f800f620 con 0x7f9608101b10 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.539+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96081a2280 con 0x7f9608101b10 2024-09-05T02:09:46.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.540+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96081a2510 con 0x7f9608101b10 2024-09-05T02:09:46.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.541+0000 7f96057fa640 1 -- 172.21.15.103:0/4046729883 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f95f801a070 con 0x7f9608101b10 2024-09-05T02:09:46.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.542+0000 7f96057fa640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f95f00a2b90 0x7f95f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.542+0000 7f96077fe640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f95f00a2b90 0x7f95f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.542+0000 7f96057fa640 1 -- 172.21.15.103:0/4046729883 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f95f80ceef0 con 0x7f9608101b10 2024-09-05T02:09:46.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.542+0000 7f96077fe640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f95f00a2b90 0x7f95f00a5050 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f95fc005d00 tx=0x7f95fc009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:46.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.546+0000 7f96057fa640 1 -- 172.21.15.103:0/4046729883 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95f8009de0 con 0x7f9608101b10 2024-09-05T02:09:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:46 smithi167 ceph-mon[35122]: pgmap v570: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 737 KiB/s wr, 432 op/s 2024-09-05T02:09:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:46 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2695409692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:09:46.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:46 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/4135187543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T02:09:46.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:46 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/292562234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T02:09:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:46 smithi139 ceph-mon[34262]: pgmap v570: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 737 KiB/s wr, 432 op/s 2024-09-05T02:09:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:46 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2695409692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:09:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:46 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/4135187543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T02:09:46.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:46 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/292562234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T02:09:46.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.712+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9608108990 con 0x7f9608101b10 2024-09-05T02:09:46.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.712+0000 7f96057fa640 1 -- 172.21.15.103:0/4046729883 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f95f809baa0 con 0x7f9608101b10 2024-09-05T02:09:46.714 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-05T02:09:46.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f95f00a2b90 msgr2=0x7f95f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f95f00a2b90 0x7f95f00a5050 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f95fc005d00 tx=0x7f95fc009550 comp rx=0 tx=0).stop 2024-09-05T02:09:46.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9608101b10 msgr2=0x7f9608075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9608101b10 0x7f9608075fa0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f95f800d950 tx=0x7f95f800de10 comp rx=0 tx=0).stop 2024-09-05T02:09:46.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 shutdown_connections 2024-09-05T02:09:46.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96081036c0 0x7f9608074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f95f00a2b90 0x7f95f00a5050 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9608102d00 0x7f96080745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 --2- 172.21.15.103:0/4046729883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9608101b10 0x7f9608075fa0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 >> 172.21.15.103:0/4046729883 conn(0x7f96080fd2b0 msgr2=0x7f96080ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:46.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 shutdown_connections 2024-09-05T02:09:46.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.714+0000 7f960e7d6640 1 -- 172.21.15.103:0/4046729883 wait complete. 2024-09-05T02:09:46.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-05T02:09:46.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.824+0000 7f368109b640 1 -- 172.21.15.103:0/2825815424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c1025c0 msgr2=0x7f367c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.824+0000 7f368109b640 1 --2- 172.21.15.103:0/2825815424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c1025c0 0x7f367c108e50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3674009900 tx=0x7f367402a410 comp rx=0 tx=0).stop 2024-09-05T02:09:46.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.824+0000 7f368109b640 1 -- 172.21.15.103:0/2825815424 shutdown_connections 2024-09-05T02:09:46.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.824+0000 7f368109b640 1 --2- 172.21.15.103:0/2825815424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c1025c0 0x7f367c108e50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.824+0000 7f368109b640 1 --2- 172.21.15.103:0/2825815424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f367c101c00 0x7f367c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.824+0000 7f368109b640 1 --2- 172.21.15.103:0/2825815424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367c100a10 0x7f367c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.824+0000 7f368109b640 1 -- 172.21.15.103:0/2825815424 >> 172.21.15.103:0/2825815424 conn(0x7f367c0fc170 msgr2=0x7f367c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.824+0000 7f368109b640 1 -- 172.21.15.103:0/2825815424 shutdown_connections 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.824+0000 7f368109b640 1 -- 172.21.15.103:0/2825815424 wait complete. 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.825+0000 7f368109b640 1 Processor -- start 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.825+0000 7f368109b640 1 -- start start 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.825+0000 7f368109b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c100a10 0x7f367c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.825+0000 7f368109b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367c101c00 0x7f367c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.825+0000 7f368109b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f367c1025c0 0x7f367c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.825+0000 7f368109b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f367c10d6f0 con 0x7f367c100a10 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.825+0000 7f368109b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f367c10d570 con 0x7f367c101c00 2024-09-05T02:09:46.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.825+0000 7f368109b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f367c10d870 con 0x7f367c1025c0 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c100a10 0x7f367c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c100a10 0x7f367c197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51650/0 (socket says 172.21.15.103:51650) 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367ad9d640 1 -- 172.21.15.103:0/2870627130 learned_addr learned my addr 172.21.15.103:0/2870627130 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367a59c640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367c101c00 0x7f367c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367b59e640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f367c1025c0 0x7f367c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367ad9d640 1 -- 172.21.15.103:0/2870627130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f367c1025c0 msgr2=0x7f367c19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367ad9d640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f367c1025c0 0x7f367c19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367ad9d640 1 -- 172.21.15.103:0/2870627130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367c101c00 msgr2=0x7f367c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367ad9d640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367c101c00 0x7f367c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367ad9d640 1 -- 172.21.15.103:0/2870627130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c19f960 con 0x7f367c100a10 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367a59c640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367c101c00 0x7f367c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f367ad9d640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c100a10 0x7f367c197d40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f366c009fd0 tx=0x7f366c00c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:46.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f3663fff640 1 -- 172.21.15.103:0/2870627130 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c011470 con 0x7f367c100a10 2024-09-05T02:09:46.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367c19fc50 con 0x7f367c100a10 2024-09-05T02:09:46.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f3663fff640 1 -- 172.21.15.103:0/2870627130 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f366c011ac0 con 0x7f367c100a10 2024-09-05T02:09:46.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f3663fff640 1 -- 172.21.15.103:0/2870627130 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c01aab0 con 0x7f367c100a10 2024-09-05T02:09:46.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.826+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367c1a01a0 con 0x7f367c100a10 2024-09-05T02:09:46.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.827+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367c107e00 con 0x7f367c100a10 2024-09-05T02:09:46.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.828+0000 7f3663fff640 1 -- 172.21.15.103:0/2870627130 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f366c011c70 con 0x7f367c100a10 2024-09-05T02:09:46.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.829+0000 7f3663fff640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f36640a2b90 0x7f36640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:09:46.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.829+0000 7f367a59c640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f36640a2b90 0x7f36640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:09:46.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.829+0000 7f3663fff640 1 -- 172.21.15.103:0/2870627130 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f366c0cfa30 con 0x7f367c100a10 2024-09-05T02:09:46.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.829+0000 7f367a59c640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f36640a2b90 0x7f36640a5050 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3670005d70 tx=0x7f3670005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:09:46.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:46.833+0000 7f3663fff640 1 -- 172.21.15.103:0/2870627130 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f366c016030 con 0x7f367c100a10 2024-09-05T02:09:47.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.049+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f367c062e90 con 0x7f367c100a10 2024-09-05T02:09:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.056+0000 7f3663fff640 1 -- 172.21.15.103:0/2870627130 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740378 (secure 0 0 0) 0x7f366c09c5e0 con 0x7f367c100a10 2024-09-05T02:09:47.071 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:09:47.071 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:09:47.071 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 18, 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: "active_gid": 14150, 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: "active_name": "x", 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addrs": { 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6800", 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 4267002703 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6801", 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 4267002703 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addr": "172.21.15.103:6801/4267002703", 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: "active_change": "2024-09-05T01:53:25.001929+0000", 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: "active_mgr_features": 4540701547738038271, 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: "available": true, 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: "standbys": [ 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24109, 2024-09-05T02:09:47.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "z", 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.074 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.075 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.076 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.077 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.078 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.079 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.081 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.082 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-05T02:09:47.084 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-05T02:09:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-05T02:09:47.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.087 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.089 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.090 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.091 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T02:09:47.092 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T02:09:47.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.094 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-05T02:09:47.095 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-05T02:09:47.097 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.098 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.099 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.100 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.100 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.100 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.100 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.100 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.100 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.101 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-05T02:09:47.105 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.106 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.107 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.108 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.109 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-05T02:09:47.110 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.111 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T02:09:47.112 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.113 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.114 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.115 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-05T02:09:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-05T02:09:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-05T02:09:47.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-05T02:09:47.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.122 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.123 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-05T02:09:47.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:47.125 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.126 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.127 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T02:09:47.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T02:09:47.129 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.130 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.132 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.134 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.135 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.136 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-05T02:09:47.137 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.138 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-05T02:09:47.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.140 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.142 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.143 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T02:09:47.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-05T02:09:47.145 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.146 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.147 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.149 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.152 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.153 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.154 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.155 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T02:09:47.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.157 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.158 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-05T02:09:47.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-05T02:09:47.159 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.159 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.159 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.159 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.159 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.159 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.159 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.159 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.160 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.161 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.162 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.163 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-05T02:09:47.165 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.166 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.167 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.170 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T02:09:47.172 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T02:09:47.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.174 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.177 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.178 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.179 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.180 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.181 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.182 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.183 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.186 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.188 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.189 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.190 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.191 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.192 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.193 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.194 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.195 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.196 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.197 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.198 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-05T02:09:47.201 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.202 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.204 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.206 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.207 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.208 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.209 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.210 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T02:09:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T02:09:47.213 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T02:09:47.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T02:09:47.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.216 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.219 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.220 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.221 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.223 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.224 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.225 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.226 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.227 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.228 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:47.230 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.231 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.232 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.233 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.234 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.236 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.237 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.239 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-05T02:09:47.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T02:09:47.242 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-05T02:09:47.243 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-05T02:09:47.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-05T02:09:47.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-05T02:09:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.249 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.250 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-05T02:09:47.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.253 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.255 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.257 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.258 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T02:09:47.259 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.260 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.262 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.263 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.264 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-05T02:09:47.265 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.266 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.266 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.266 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.266 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.266 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.266 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.269 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.270 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-05T02:09:47.272 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-05T02:09:47.273 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.276 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-05T02:09:47.277 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.278 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.279 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.282 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.283 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-05T02:09:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-05T02:09:47.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-05T02:09:47.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-05T02:09:47.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.289 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.290 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.291 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.293 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.294 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.295 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.299 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.301 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.303 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.304 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.306 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.309 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.310 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.311 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.313 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.315 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.316 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.317 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-05T02:09:47.318 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.319 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.321 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-05T02:09:47.322 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.323 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.324 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.326 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.327 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.328 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.329 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.330 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.331 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.332 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.334 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.335 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.336 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.337 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.338 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.339 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-05T02:09:47.340 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.341 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.343 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.345 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.346 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-05T02:09:47.347 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-05T02:09:47.348 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.349 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.351 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.352 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.353 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.354 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.355 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.357 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.358 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.359 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.361 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.365 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.366 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.368 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.369 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.370 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.371 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.373 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.374 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.376 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.377 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.380 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.384 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-05T02:09:47.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T02:09:47.387 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-05T02:09:47.388 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-05T02:09:47.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.390 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.392 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.393 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.394 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-05T02:09:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-05T02:09:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-05T02:09:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.440 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:47.441 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.443 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.444 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.445 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.448 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.450 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.451 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.452 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.453 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.454 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.455 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.457 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.458 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.459 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.460 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.460 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.460 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.460 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.460 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.460 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.460 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.460 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.460 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.461 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.462 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.463 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.466 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.467 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.468 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.469 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.470 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.471 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-05T02:09:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.473 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T02:09:47.475 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T02:09:47.476 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.477 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.478 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.479 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T02:09:47.480 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.481 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.482 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.483 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.484 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.485 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.486 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.487 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.489 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-05T02:09:47.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-05T02:09:47.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.493 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.494 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.495 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.496 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.497 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.498 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-05T02:09:47.500 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.501 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.502 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.503 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.504 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.505 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.506 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-05T02:09:47.507 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.508 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.509 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.510 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.512 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.513 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.514 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.515 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.516 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.517 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.518 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.519 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.520 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.521 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24113, 2024-09-05T02:09:47.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "y", 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.523 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.524 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.525 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.526 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.527 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.528 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-05T02:09:47.529 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-05T02:09:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-05T02:09:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-05T02:09:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-05T02:09:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-05T02:09:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-05T02:09:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-05T02:09:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-05T02:09:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-05T02:09:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-05T02:09:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-05T02:09:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-05T02:09:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-05T02:09:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-05T02:09:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-05T02:09:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-05T02:09:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-05T02:09:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T02:09:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T02:09:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-05T02:09:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-05T02:09:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-05T02:09:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T02:09:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-05T02:09:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-05T02:09:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T02:09:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-05T02:09:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-05T02:09:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-05T02:09:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.692 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.693 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.694 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.695 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.696 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-05T02:09:47.697 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-05T02:09:47.698 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-05T02:09:47.699 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.700 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.701 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.702 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.703 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.704 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.705 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.706 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-05T02:09:47.707 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-05T02:09:47.708 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-05T02:09:47.709 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.710 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-05T02:09:47.711 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.712 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T02:09:47.713 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T02:09:47.714 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.715 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.716 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.717 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.718 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.719 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.720 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.721 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.722 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.723 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.724 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.725 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.726 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.727 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.728 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.729 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.730 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.731 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.732 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-05T02:09:47.733 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-05T02:09:47.734 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-05T02:09:47.735 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.736 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.737 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.738 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.739 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.740 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.741 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.742 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.743 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.744 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.745 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.746 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.749 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.750 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.751 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.754 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.755 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.756 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.757 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.758 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.759 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.760 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.762 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.763 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.764 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.765 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.766 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.767 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.768 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.769 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.770 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.771 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.772 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.773 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.774 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.775 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.776 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.777 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.778 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.779 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.782 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.783 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.785 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.786 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.787 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-05T02:09:47.788 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.789 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.790 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.791 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.792 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.793 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.794 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.795 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-05T02:09:47.796 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.797 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.798 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.799 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.800 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.801 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.802 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.803 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.804 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.805 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.808 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.809 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.810 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.811 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.812 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.813 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.814 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-05T02:09:47.815 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T02:09:47.816 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.817 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.818 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.819 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.820 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.821 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.822 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.823 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.824 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-05T02:09:47.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.826 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.827 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.828 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.829 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.830 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.834 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.836 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.837 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.838 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-05T02:09:47.839 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-05T02:09:47.840 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.841 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.842 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.843 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.844 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.845 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.846 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.847 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.848 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.849 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-05T02:09:47.850 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-05T02:09:47.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.852 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.853 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.854 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.855 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.856 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.857 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.858 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.859 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.860 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.861 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.862 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.863 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.864 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.865 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.866 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.869 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.872 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.873 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.876 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.877 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.878 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.879 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.880 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.881 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.882 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-05T02:09:47.884 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-05T02:09:47.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.886 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.888 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.889 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.890 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.891 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.892 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.893 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.894 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.895 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.899 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.900 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.901 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.902 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.903 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.905 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.906 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.907 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.908 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.909 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-05T02:09:47.910 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.911 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.912 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.913 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.914 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.915 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.917 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.918 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.919 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.920 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.922 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.923 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.924 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.925 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.926 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.927 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-05T02:09:47.928 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-05T02:09:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.942 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.943 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.944 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.946 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.947 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.949 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.950 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-05T02:09:47.951 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.952 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.953 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.954 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.955 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.956 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.957 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.958 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.959 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.960 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.962 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.963 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.964 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.965 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:47.966 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.967 INFO:teuthology.orchestra.run.smithi103.stdout: "modules": [ 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard", 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "iostat", 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "nfs", 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "restful" 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:47.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.969 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.970 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.971 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.973 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.975 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.976 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.977 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.978 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-05T02:09:47.979 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.980 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.981 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.982 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:47.983 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.984 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T02:09:47.985 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T02:09:47.986 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.987 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.988 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-05T02:09:47.989 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.990 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-05T02:09:47.991 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.992 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.994 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:47.995 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:47.996 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:47.997 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-05T02:09:47.998 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:47.999 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.000 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.001 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.003 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.004 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.005 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.008 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.009 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.010 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.011 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.013 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.014 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-05T02:09:48.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.018 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.019 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.020 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.021 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-05T02:09:48.022 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-05T02:09:48.023 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.024 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.025 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.026 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.027 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.028 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.029 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.030 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.031 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:48.032 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.033 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.034 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.036 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.037 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.038 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.039 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-05T02:09:48.040 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-05T02:09:48.041 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.043 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.044 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.045 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.046 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.047 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T02:09:48.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T02:09:48.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.050 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.051 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T02:09:48.054 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T02:09:48.055 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T02:09:48.056 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T02:09:48.057 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T02:09:48.058 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.059 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.060 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.061 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.062 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.063 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.064 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.065 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-05T02:09:48.066 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-05T02:09:48.067 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-05T02:09:48.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.069 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.070 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.072 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.073 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.074 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.075 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.076 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.078 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.079 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.080 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.081 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.083 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.084 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T02:09:48.086 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T02:09:48.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T02:09:48.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.089 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.090 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.094 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.095 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.096 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.097 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.098 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.099 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.100 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.102 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.103 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.104 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.105 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.106 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T02:09:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T02:09:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T02:09:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T02:09:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T02:09:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T02:09:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-05T02:09:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-05T02:09:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-05T02:09:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-05T02:09:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-05T02:09:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-05T02:09:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-05T02:09:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-05T02:09:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-05T02:09:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-05T02:09:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T02:09:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-05T02:09:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-05T02:09:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-05T02:09:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-05T02:09:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-05T02:09:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-05T02:09:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-05T02:09:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-05T02:09:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-05T02:09:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-05T02:09:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-05T02:09:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-05T02:09:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-05T02:09:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-05T02:09:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-05T02:09:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-05T02:09:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-05T02:09:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-05T02:09:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-05T02:09:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-05T02:09:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "services": { 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard": "https://172.21.15.103:8443/" 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "always_on_modules": { 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "octopus": [ 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-05T02:09:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "pacific": [ 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-05T02:09:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "quincy": [ 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-05T02:09:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "reef": [ 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-05T02:09:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "squid": [ 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "last_failure_osd_epoch": 3, 2024-09-05T02:09:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "active_clients": [ 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3372898792 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "libcephsqlite", 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-05T02:09:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2954782588 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2062660682 2024-09-05T02:09:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 389701921 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-05T02:09:48.422 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:09:48.424 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:47 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/3071369242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T02:09:48.424 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:47 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/4046729883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T02:09:48.424 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:47 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2870627130' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T02:09:48.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:47 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/3071369242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T02:09:48.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:47 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/4046729883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T02:09:48.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:47 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2870627130' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T02:09:48.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f36640a2b90 msgr2=0x7f36640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:48.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f36640a2b90 0x7f36640a5050 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3670005d70 tx=0x7f3670005d00 comp rx=0 tx=0).stop 2024-09-05T02:09:48.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c100a10 msgr2=0x7f367c197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:09:48.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c100a10 0x7f367c197d40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f366c009fd0 tx=0x7f366c00c270 comp rx=0 tx=0).stop 2024-09-05T02:09:48.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 shutdown_connections 2024-09-05T02:09:48.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f36640a2b90 0x7f36640a5050 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:48.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f367c1025c0 0x7f367c19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367c101c00 0x7f367c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 --2- 172.21.15.103:0/2870627130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f367c100a10 0x7f367c197d40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:09:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 >> 172.21.15.103:0/2870627130 conn(0x7f367c0fc170 msgr2=0x7f367c104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:09:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 shutdown_connections 2024-09-05T02:09:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:09:47.062+0000 7f368109b640 1 -- 172.21.15.103:0/2870627130 wait complete. 2024-09-05T02:09:48.432 DEBUG:tasks.check_counter:Getting stats from a 2024-09-05T02:09:48.432 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-05T02:09:48.432 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:47 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/3071369242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T02:09:48.432 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:47 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/4046729883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T02:09:48.433 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:47 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2870627130' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T02:09:48.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:48 smithi167 ceph-mon[35122]: pgmap v571: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 524 KiB/s wr, 299 op/s 2024-09-05T02:09:48.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:48 smithi139 ceph-mon[34262]: pgmap v571: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 524 KiB/s wr, 299 op/s 2024-09-05T02:09:48.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:48 smithi103 ceph-mon[30902]: pgmap v571: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 524 KiB/s wr, 299 op/s 2024-09-05T02:09:48.690 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:09:49.201 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:09:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:09:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 907, 2024-09-05T02:09:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 907, 2024-09-05T02:09:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 8666221, 2024-09-05T02:09:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 287603, 2024-09-05T02:09:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 7, 2024-09-05T02:09:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.079063507, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.023059494, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.026861878, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.022594549, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 906, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.029099380, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000032118 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 681, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 801, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 18291967, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 496676, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.069526208, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.018989511, 2024-09-05T02:09:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.028508433, 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.017289815, 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 799, 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.022799737, 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000028535 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 280368 2024-09-05T02:09:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 315, 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 298, 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 4802774, 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 140368, 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.033619610, 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.013397967, 2024-09-05T02:09:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.020720612, 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.002921872, 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 297, 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.014891741, 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000050140 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 575712, 2024-09-05T02:09:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 99664 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2917, 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.063325752, 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000021709 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 1779, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 1779, 2024-09-05T02:09:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 1779, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-05T02:09:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:09:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-05T02:09:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-05T02:09:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-05T02:09:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 31269, 2024-09-05T02:09:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 37, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 4, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 34897738, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 34896600, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 31269 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 13, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 1948, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 1935, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 11, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 111, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 100, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 1779, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 4493, 2024-09-05T02:09:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 2714, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 54292, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-05T02:09:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-05T02:09:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-05T02:09:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 2, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1489, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-05T02:09:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:09:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-05T02:09:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:09:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:09:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:09:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:09:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:09:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:09:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:09:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:09:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:09:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 215147, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 66, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8952, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 168, 2024-09-05T02:09:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:09:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:09:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-05T02:09:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-05T02:09:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 1296008, 2024-09-05T02:09:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 7244, 2024-09-05T02:09:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-05T02:09:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-05T02:09:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-05T02:09:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 1711, 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 1711, 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1711, 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.074979495, 2024-09-05T02:09:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000628275 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1711, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1712 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 1711, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 1711, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 970, 2024-09-05T02:09:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 739, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-09-05T02:09:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-05T02:09:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-05T02:09:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 90, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 4, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 8, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 8, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:09:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-05T02:09:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 1899, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 31577531, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 1899, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1899, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 31577531, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 1711, 2024-09-05T02:09:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 30703544, 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1711, 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 30703544, 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-05T02:09:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 1711, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 1711, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1711, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1711, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x56439ebe4b40": { 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-05T02:09:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-05T02:09:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-05T02:09:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-05T02:09:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:49.249 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:09:50.003 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-05T02:09:50.004 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-05T02:09:50.004 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-05T02:09:50.004 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-05T02:09:50.004 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-05T02:09:50.004 DEBUG:tasks.check_counter:Getting stats from d 2024-09-05T02:09:50.004 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-05T02:09:50.253 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:09:50.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:50 smithi167 ceph-mon[35122]: pgmap v572: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 473 KiB/s wr, 275 op/s 2024-09-05T02:09:50.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:50 smithi139 ceph-mon[34262]: pgmap v572: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 473 KiB/s wr, 275 op/s 2024-09-05T02:09:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:50 smithi103 ceph-mon[30902]: pgmap v572: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 473 KiB/s wr, 275 op/s 2024-09-05T02:09:50.790 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.001594958, 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000304141, 2024-09-05T02:09:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T02:09:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 5, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 128, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3152, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 101120, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.012030109, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.007181355, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.003900312, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 127, 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.009068523, 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000071405 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 101120 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 169, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 166, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 463952, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 101328, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.022739510, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.010799275, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.015206532, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 165, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.007023495, 2024-09-05T02:09:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000042566 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 463952, 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 101328 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-05T02:09:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:09:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:09:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:09:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148967, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 29, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:09:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-05T02:09:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 170, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 440095, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 170, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 170, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 440095, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:50.802 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:50.803 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:50.803 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:50.803 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:09:51.571 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-05T02:09:51.571 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-05T02:09:51.571 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-05T02:09:51.571 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-05T02:09:51.571 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-05T02:09:51.571 DEBUG:tasks.check_counter:Getting stats from g 2024-09-05T02:09:51.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-05T02:09:51.817 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:09:52.309 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:09:52.309 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:09:52.309 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 722, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 722, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 12814189, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 228773, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 7, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.074523915, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.025329853, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.024730564, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.018664116, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 721, 2024-09-05T02:09:52.310 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.028645425, 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000039730 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 911, 2024-09-05T02:09:52.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1032, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 25397952, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 569981, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.087081294, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.022531685, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.038425365, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.020699173, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1030, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.027274690, 2024-09-05T02:09:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000026480 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 280544 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 274, 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 257, 2024-09-05T02:09:52.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 6948464, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 127663, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.033579139, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.011141598, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.024257987, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.002146197, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 256, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.011850326, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000046290 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.314 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 575984, 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 100304 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-05T02:09:52.315 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2962, 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.062753943, 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000021186 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:52.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-05T02:09:52.317 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-05T02:09:52.318 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-05T02:09:52.318 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 2222, 2024-09-05T02:09:52.318 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-09-05T02:09:52.318 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 2222, 2024-09-05T02:09:52.318 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-05T02:09:52.318 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 2222, 2024-09-05T02:09:52.318 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-05T02:09:52.318 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-05T02:09:52.319 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:09:52.320 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-05T02:09:52.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-05T02:09:52.322 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-05T02:09:52.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 41248, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-05T02:09:52.324 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 49, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 5, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 48293114, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 48291976, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.325 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 41248 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 13, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 2369, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 2356, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 11, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 144, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 133, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 2222, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 6306, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 4084, 2024-09-05T02:09:52.326 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 53668, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-05T02:09:52.327 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.329 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-05T02:09:52.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.332 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.332 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.334 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-05T02:09:52.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.334 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.334 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-05T02:09:52.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.334 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.335 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-05T02:09:52.336 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.336 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.336 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.336 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-05T02:09:52.336 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.336 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.336 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.337 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-05T02:09:52.337 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.337 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.337 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.337 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-05T02:09:52.338 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.338 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.338 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.338 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-05T02:09:52.338 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.338 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.338 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.339 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-05T02:09:52.339 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.339 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.339 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.339 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-05T02:09:52.339 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.339 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.340 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.342 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 1, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 0, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1495, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:09:52.343 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:09:52.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:09:52.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:09:52.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-05T02:09:52.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:09:52.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-05T02:09:52.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:09:52.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:09:52.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-05T02:09:52.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 222544, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 70, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-05T02:09:52.347 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8952, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 168, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 1607880, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 9016, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-05T02:09:52.348 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 1714, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 1714, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1714, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.991981127, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000578752 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-05T02:09:52.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1714, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1715 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 1714, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 1714, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 960, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 752, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-05T02:09:52.350 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:09:52.351 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-05T02:09:52.352 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 90, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-05T02:09:52.353 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 4, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 9, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 9, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-05T02:09:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 1903, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 44976762, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 1903, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1903, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 44976762, 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:52.356 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:52.357 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 1714, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 44101219, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1714, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 44101219, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:52.358 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 1714, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 1714, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1714, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1714, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:52.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:52.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x55be5d546b40": { 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:52.361 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:52.362 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:09:52.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:52 smithi167 ceph-mon[35122]: pgmap v573: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 454 KiB/s wr, 193 op/s 2024-09-05T02:09:52.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:52 smithi139 ceph-mon[34262]: pgmap v573: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 454 KiB/s wr, 193 op/s 2024-09-05T02:09:52.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:52 smithi103 ceph-mon[30902]: pgmap v573: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 454 KiB/s wr, 193 op/s 2024-09-05T02:09:53.168 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-05T02:09:53.168 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-05T02:09:53.168 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-05T02:09:53.168 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-05T02:09:53.168 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-05T02:09:53.168 DEBUG:tasks.check_counter:Getting stats from j 2024-09-05T02:09:53.168 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-05T02:09:53.415 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:09:53.939 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-05T02:09:53.939 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:09:53.939 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 175, 2024-09-05T02:09:53.939 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 170, 2024-09-05T02:09:53.939 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 467488, 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 103056, 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.025044397, 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.010605568, 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.017109326, 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 170, 2024-09-05T02:09:53.940 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.011463631, 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000067433 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 467488, 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 103056 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1, 2024-09-05T02:09:53.941 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 127, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 160, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 100848, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.009772006, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.007284844, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001385108, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 125, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.004860169, 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000038881 2024-09-05T02:09:53.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 100848 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-05T02:09:53.943 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.000352496, 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000285258, 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-05T02:09:53.945 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:09:53.946 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-05T02:09:53.947 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 149007, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 30, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:09:53.948 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-05T02:09:53.949 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 172, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 440355, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 172, 2024-09-05T02:09:53.950 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 172, 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 440355, 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-05T02:09:53.951 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-05T02:09:54.576 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:54 smithi103 ceph-mon[30902]: pgmap v574: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 99 KiB/s wr, 107 op/s 2024-09-05T02:09:54.605 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-05T02:09:54.606 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-05T02:09:54.606 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-05T02:09:54.606 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-05T02:09:54.606 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-05T02:09:54.606 DEBUG:tasks.check_counter:Getting stats from b 2024-09-05T02:09:54.606 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-05T02:09:54.634 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:54 smithi139 ceph-mon[34262]: pgmap v574: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 99 KiB/s wr, 107 op/s 2024-09-05T02:09:54.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:54 smithi167 ceph-mon[35122]: pgmap v574: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 99 KiB/s wr, 107 op/s 2024-09-05T02:09:54.862 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T02:09:55.402 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 233245, 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 104935, 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 82425300, 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 56952115, 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 16, 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 7.967252770, 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 3.036497699, 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 6.575696089, 2024-09-05T02:09:55.403 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.166567963, 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 104936, 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 7.107020683, 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000067727 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 322541, 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.038439651, 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000119 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 55376, 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 29856 2024-09-05T02:09:55.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 10137, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 10720, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 3664579, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 21527243, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 11, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 6, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.714858465, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.292020574, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.278727080, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.150833925, 2024-09-05T02:09:55.405 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 10719, 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 5.684836632, 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000530351 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 8364, 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.001585497, 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000189 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 615888, 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 107376 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.406 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 13181, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 14286, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 3870630, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 15425122, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 95, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 9, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.959711226, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.455848215, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.450970942, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.100395002, 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:55.407 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 14282, 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 2.706869629, 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000189530 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 18997, 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.003068655, 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000161 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 299024 2024-09-05T02:09:55.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6247, 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 5.838164302, 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000934554 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.409 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 5319, 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.573991119, 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000107913 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 49123, 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 35334, 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 35334, 2024-09-05T02:09:55.410 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 130.515941698, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.003693777 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 13789, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 941, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 9, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 154, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 4, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 12, 2024-09-05T02:09:55.411 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 138, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 147, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 1235, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 2, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 2, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 4, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 96836, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 52814, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 13785, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 1342, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 6, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 4302, 2024-09-05T02:09:55.412 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 102, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 726, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 39, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 3005, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 4, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 234, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-09-05T02:09:55.413 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 76, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 5075, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 4555, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 5, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 136268, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 1263, 2024-09-05T02:09:55.414 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 117, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 8133 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 2674, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 9562, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 7156, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 10748, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 2409, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 2, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 2584, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 2582, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-09-05T02:09:55.415 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 5, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 4499, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 109, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 9, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-09-05T02:09:55.416 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 33812, 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 32523, 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 32523, 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 1289, 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 2541, 2024-09-05T02:09:55.417 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 57, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 54, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 54, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 3, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 1, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 4, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 41793212, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 43719403, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 0, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2879, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 10.416570269, 2024-09-05T02:09:55.418 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.003618120 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 0 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 32, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 2935, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 2903, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 51, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 2639, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 2588, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 462, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 7159, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 6697, 2024-09-05T02:09:55.419 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 2, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 2829, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 2827, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 86348, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 231712 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 56656, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 641, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 49123, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 49, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 3987, 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-09-05T02:09:55.420 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1086, 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 58.638453099, 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.053994892 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 425, 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.183285457, 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000431259 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 30244, 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 17.533228417, 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000579725 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.423 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.424 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.424 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-09-05T02:09:55.424 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.424 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.424 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 21, 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.025084774, 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001194513 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.425 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 248, 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.698403535, 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002816143 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.426 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 236, 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.478395792, 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002027100 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 699, 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 49.143710228, 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.070305737 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 10, 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.081442490, 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.008144249 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.428 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2, 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.001378424, 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000689212 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.429 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.430 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 8, 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.021627885, 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002703485 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3, 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.002463708, 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000821236 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2352, 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 3.708467889, 2024-09-05T02:09:55.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001576729 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 3, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 2, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 1, 2024-09-05T02:09:55.432 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 9235, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 9235, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 433, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 360, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 360, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:09:55.433 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:09:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-09-05T02:09:55.435 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 302823, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 107, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:09:55.436 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8952, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 168, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 616353, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 7489, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-09-05T02:09:55.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 15028, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 44863437, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 37, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 15028, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 15028, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 247.449725510, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.016465911 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-09-05T02:09:55.438 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 15028, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 24056 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 15028, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 192, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 14836, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 830, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 4559, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 192, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 3071, 2024-09-05T02:09:55.439 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 237, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 9, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 5682, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 6959, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-09-05T02:09:55.440 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 5, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-09-05T02:09:55.441 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 90, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 10, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 12, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 12, 2024-09-05T02:09:55.442 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 2089, 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 0, 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 423 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 1, 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-09-05T02:09:55.443 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 4508 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 5, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 5153, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 2582, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-05T02:09:55.444 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 256559, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 66849137, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 256559, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 256559, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 66849137, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.445 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 15028, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 45144787, 2024-09-05T02:09:55.446 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 15028, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 45144787, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:09:55.447 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 15028, 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 15028, 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 15028, 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 15028, 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:09:55.448 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 2582, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 281350, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 2582, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 192, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 281350, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.449 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x564ea5b26960": { 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 33812, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 39525099, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 33812, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:09:55.450 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2879, 2024-09-05T02:09:55.451 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 39525099, 2024-09-05T02:09:55.451 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:55.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:55.451 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:55.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:55.451 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:55.451 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:55.451 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-05T02:09:56.398 INFO:tasks.check_counter:Daemon mds.b mds.exported=102 2024-09-05T02:09:56.398 INFO:tasks.check_counter:Daemon mds.b mds.imported=39 2024-09-05T02:09:56.399 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=2674 2024-09-05T02:09:56.399 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=9562 2024-09-05T02:09:56.399 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=9 2024-09-05T02:09:56.399 DEBUG:tasks.check_counter:Getting stats from e 2024-09-05T02:09:56.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-05T02:09:56.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:56 smithi167 ceph-mon[35122]: pgmap v575: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 77 op/s 2024-09-05T02:09:56.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:56 smithi139 ceph-mon[34262]: pgmap v575: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 77 op/s 2024-09-05T02:09:56.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:56 smithi103 ceph-mon[30902]: pgmap v575: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 77 op/s 2024-09-05T02:09:56.683 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T02:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 290593, 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 121427, 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 101671530, 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 61320864, 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 16, 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 6, 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 9.124583652, 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 3.455286718, 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 7.745245578, 2024-09-05T02:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.114577621, 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 121424, 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 2.291722317, 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000018873 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 398493, 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.044817379, 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000112 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T02:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 13312, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 12893, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 4248059, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 11648129, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 9, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 6, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.790933597, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.339656794, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.476850195, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.052818168, 2024-09-05T02:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 12894, 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.408648171, 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000031692 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 21595, 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.003289783, 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000152 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 589648, 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 108496 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 5950, 2024-09-05T02:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 6278, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 1613199, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 17225093, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 11, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 8, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.437470697, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.187897433, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.180277634, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.076848512, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6276, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.228665483, 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000036434 2024-09-05T02:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6344, 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.001020073, 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000160 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 282864 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-09-05T02:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 5703, 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 4.003635618, 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000702022 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-05T02:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 4658, 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.461051394, 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000098980 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 71231, 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 60808, 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 60808, 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 198.948788702, 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.003271753 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-09-05T02:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 10423, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 74, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 7, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 17, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 2, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 4, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 11, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 14, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 669, 2024-09-05T02:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 4, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 146486, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 51282, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 10423, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 1220, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 1, 2024-09-05T02:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 664, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 90, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 188, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 41, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 2928, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 2690, 2024-09-05T02:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 59616945, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 174, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 4597, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 3810, 2024-09-05T02:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 175472, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 712, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 116, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 7000 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 2333, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 9267, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 6955, 2024-09-05T02:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 10118, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 3296, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 2212, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 2212, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-09-05T02:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 5, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 4470, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 92, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 7, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 41248, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-09-05T02:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 41248, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 49, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 49, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 5, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 48293114, 2024-09-05T02:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 0, 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2867, 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 12.093844275, 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.004218292 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 0 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 24, 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 2447, 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 2423, 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 29, 2024-09-05T02:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 2354, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 2325, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 105, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 7050, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 6945, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 2357, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 2357, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 82168, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 223520 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 79457, 2024-09-05T02:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 1506, 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 71231, 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 95, 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 5072, 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 708, 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 25.334558184, 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.035783274 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 56995, 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 30.760891564, 2024-09-05T02:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000539712 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-09-05T02:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-09-05T02:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-09-05T02:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 203, 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 3.402618360, 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.016761666 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 145, 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.110273813, 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000760509 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 708, 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 136.141112045, 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.192289706 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2049, 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 3.199334736, 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001561412 2024-09-05T02:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 1, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 0, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 1, 2024-09-05T02:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 17319, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 17319, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 432, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 279745, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 99, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8952, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 168, 2024-09-05T02:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 177213, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 1651, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-09-05T02:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 8132, 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 45412229, 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 32, 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 8132, 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 8132, 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 27.991404849, 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.003442130 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 8132, 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 8647 2024-09-05T02:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 8132, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 177, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 7955, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 0, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 10, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 177, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 3044, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 128, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-09-05T02:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 7, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 4446, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 20, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-09-05T02:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 90, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 4, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 12, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 12, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 593, 2024-09-05T02:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 0, 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 222 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 1, 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 4479 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-09-05T02:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 5, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 4424, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 2212, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 309851, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 79631939, 2024-09-05T02:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 309851, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 309851, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 79631939, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 8132, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 45653337, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 8132, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 45653337, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 8132, 2024-09-05T02:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 8132, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 8132, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 8132, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 2212, 2024-09-05T02:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 241108, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 2212, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 177, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 241108, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x55f16a724960": { 2024-09-05T02:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 41248, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 44098810, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 41248, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2867, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 44098810, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-05T02:09:57.891 INFO:tasks.check_counter:Daemon mds.e mds.exported=90 2024-09-05T02:09:57.891 INFO:tasks.check_counter:Daemon mds.e mds.imported=41 2024-09-05T02:09:57.891 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=2333 2024-09-05T02:09:57.891 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=9267 2024-09-05T02:09:57.891 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=7 2024-09-05T02:09:57.892 DEBUG:tasks.check_counter:Getting stats from h 2024-09-05T02:09:57.892 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-05T02:09:58.137 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T02:09:58.559 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:09:58 smithi139 ceph-mon[34262]: pgmap v576: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-05T02:09:58.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:09:58 smithi167 ceph-mon[35122]: pgmap v576: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-05T02:09:58.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:09:58 smithi103 ceph-mon[30902]: pgmap v576: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-05T02:09:58.660 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 10236, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 9325, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 2936071, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 11076528, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 11, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.675368677, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.303515934, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.347202510, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.067395216, 2024-09-05T02:09:58.661 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 9326, 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.312079592, 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000033463 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 14328, 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.002209856, 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000154 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.662 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 42072, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 33698, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 17586999, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 18602809, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 11, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 9, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 1.990298209, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.782127320, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 1.410383975, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.073471903, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 33697, 2024-09-05T02:09:58.663 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.857276219, 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000025440 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 69514, 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.010756433, 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000154 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 600384, 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 107216 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:09:58.664 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 164465, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 62496, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 55512287, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 29729919, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 14, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 7, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 5.385413406, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 2.015254415, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 4.793152328, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.039598876, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 62492, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.087847120, 2024-09-05T02:09:58.665 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000017407 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 220585, 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.025900619, 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000117 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 284304 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.666 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 4875, 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 3.123480958, 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000640714 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3581, 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.292098480, 2024-09-05T02:09:58.667 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000081568 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 47275, 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 37073, 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 37073, 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 220.241452020, 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.005940750 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 10202, 2024-09-05T02:09:58.668 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 83, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 8, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 15, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 0, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 1, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 14, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 14, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 370, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 1, 2024-09-05T02:09:58.669 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 1, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 3, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 98086, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 41483, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 10202, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 1313, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 14, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 3680, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 74, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 123, 2024-09-05T02:09:58.670 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 18, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 2081, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 2690, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 59616945, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:09:58.671 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 32, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 3607, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 3143, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 113151, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 660, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 89, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 5791 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 2054, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 10239, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 7437, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 11383, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 2632, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 1677, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 1677, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-09-05T02:09:58.673 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 5, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 3388, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 74, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 8, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-09-05T02:09:58.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 22253, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 22253, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 26, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 26, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 3, 2024-09-05T02:09:58.675 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 27747860, 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 0, 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2422, 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 9.333234791, 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.003853523 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 0 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-09-05T02:09:58.676 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 22, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 1803, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 1781, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 26, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 3058, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 3032, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 38, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 5100, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 5062, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 1, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 1696, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 1695, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 74940, 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 223520 2024-09-05T02:09:58.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 54154, 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 92, 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 47275, 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 9, 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 3064, 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 658, 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 78.841984460, 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.119820645 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-09-05T02:09:58.678 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3, 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.039005449, 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.013001816 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.679 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 33825, 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 15.935098171, 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000471104 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.682 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-09-05T02:09:58.682 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.682 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.682 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.683 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 317, 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.955293967, 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.006168119 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 128, 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.093482923, 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000730335 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.684 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 666, 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 121.060857766, 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.181773059 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.687 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.688 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1476, 2024-09-05T02:09:58.689 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 2.315729284, 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001568922 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 1, 2024-09-05T02:09:58.690 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 0, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 1, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 10879, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 10879, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 435, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:09:58.691 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:09:58.692 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-09-05T02:09:58.693 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 288043, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 105, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-09-05T02:09:58.694 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8952, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 168, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 114277, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 751, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-09-05T02:09:58.695 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 6513, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 24873799, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 28, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 6513, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6513, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 22.406522764, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.003440276 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-09-05T02:09:58.696 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6513, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 6981 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 6513, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 138, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 6375, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 0, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 10, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 138, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 2560, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 109, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-09-05T02:09:58.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 8, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 3371, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 20, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-09-05T02:09:58.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-09-05T02:09:58.699 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 90, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 4, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 12, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 12, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-09-05T02:09:58.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 580, 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 0, 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 185 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 1, 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 3399 2024-09-05T02:09:58.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 5, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 3354, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 1677, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:58.702 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 216769, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 56511942, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 216769, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 216769, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 56511942, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-09-05T02:09:58.703 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 6513, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 25056592, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 6513, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 25056592, 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:58.704 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.705 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.705 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.705 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:58.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.705 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-09-05T02:09:58.705 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:58.705 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-09-05T02:09:58.705 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:58.705 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 6513, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 6513, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 6513, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 6513, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.706 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 1677, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 182793, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 1677, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 138, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 182793, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.707 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x55f3c3286960": { 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 22253, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 23553556, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 22253, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:09:58.708 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2422, 2024-09-05T02:09:58.709 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 23553556, 2024-09-05T02:09:58.709 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:09:58.709 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:09:58.709 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:09:58.709 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:09:58.709 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:58.709 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:09:58.709 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-05T02:09:59.485 INFO:tasks.check_counter:Daemon mds.h mds.exported=74 2024-09-05T02:09:59.485 INFO:tasks.check_counter:Daemon mds.h mds.imported=18 2024-09-05T02:09:59.485 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=2054 2024-09-05T02:09:59.485 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=10239 2024-09-05T02:09:59.485 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=8 2024-09-05T02:09:59.485 DEBUG:tasks.check_counter:Getting stats from k 2024-09-05T02:09:59.485 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-05T02:09:59.729 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T02:10:00.244 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-05T02:10:00.244 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:10:00.244 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 285, 2024-09-05T02:10:00.244 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 266, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 4797514, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 132908, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 2, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.035915361, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.012303712, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.026189928, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.002155061, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 266, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.015769155, 2024-09-05T02:10:00.245 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000059282 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 581152, 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 105872 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 523, 2024-09-05T02:10:00.246 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 651, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 11101862, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 451009, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.052238912, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.015775915, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.018708461, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.013085538, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 649, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.020846589, 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000032121 2024-09-05T02:10:00.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 285008 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 1123, 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 1125, 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 8723137, 2024-09-05T02:10:00.248 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 357172, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.094384645, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.030481991, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.027546242, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.028543421, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1123, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.035021368, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000031185 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:00.249 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-05T02:10:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2984, 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.067746944, 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000022703 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:10:00.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-09-05T02:10:00.252 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 1687, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 0, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 1687, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 1687, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 1, 2024-09-05T02:10:00.253 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-09-05T02:10:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:10:00.255 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.256 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-09-05T02:10:00.257 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-09-05T02:10:00.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 22253, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-09-05T02:10:00.259 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 26, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 3, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 27747860, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 27746722, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.260 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 22253 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 13, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 1740, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 1727, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 11, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 127, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 116, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 1687, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 4739, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 3052, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-09-05T02:10:00.261 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 53628, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215328 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.262 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.265 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.266 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.267 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-09-05T02:10:00.268 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.269 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.269 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.269 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-09-05T02:10:00.269 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.269 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.269 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.270 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.271 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.272 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.273 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-09-05T02:10:00.274 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 1, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 0, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 1, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 1542, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:10:00.275 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-09-05T02:10:00.276 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-09-05T02:10:00.277 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 207128, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 63, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-09-05T02:10:00.278 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8952, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 168, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 1231240, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 6876, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-09-05T02:10:00.279 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 1730, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 21, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 3, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 1730, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1730, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.948979677, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000548543 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.280 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1730, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1731 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 1730, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 1730, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 978, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 750, 2024-09-05T02:10:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 1, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:10:00.282 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-09-05T02:10:00.283 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-09-05T02:10:00.284 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 90, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 4, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 7, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 7, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-09-05T02:10:00.285 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-09-05T02:10:00.286 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 1927, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 24436094, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 1927, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:10:00.287 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1927, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 24436094, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:10:00.288 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 1730, 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 23557149, 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1730, 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 23557149, 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.289 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 1730, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 1730, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1730, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 1730, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.290 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:10:00.291 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x55c901ef2b40": { 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-05T02:10:00.292 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:10:00.293 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-05T02:10:00.294 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-05T02:10:00.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:00 smithi139 ceph-mon[34262]: pgmap v577: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-09-05T02:10:00.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:00 smithi139 ceph-mon[34262]: overall HEALTH_OK 2024-09-05T02:10:00.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:00 smithi167 ceph-mon[35122]: pgmap v577: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-09-05T02:10:00.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:00 smithi167 ceph-mon[35122]: overall HEALTH_OK 2024-09-05T02:10:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:00 smithi103 ceph-mon[30902]: pgmap v577: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-09-05T02:10:00.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:00 smithi103 ceph-mon[30902]: overall HEALTH_OK 2024-09-05T02:10:01.636 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-05T02:10:01.637 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-05T02:10:01.637 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-05T02:10:01.637 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-05T02:10:01.637 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-05T02:10:01.637 DEBUG:tasks.check_counter:Getting stats from c 2024-09-05T02:10:01.637 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-05T02:10:01.898 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T02:10:02.423 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-05T02:10:02.423 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:10:02.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 12654, 2024-09-05T02:10:02.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 12168, 2024-09-05T02:10:02.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 3970253, 2024-09-05T02:10:02.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 12221245, 2024-09-05T02:10:02.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 11, 2024-09-05T02:10:02.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.675205035, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.264322888, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.387426096, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.069499633, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 12169, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.298829184, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000024556 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 19085, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.002697739, 2024-09-05T02:10:02.424 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000141 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 3642, 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 3769, 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 927697, 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 18397771, 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 15, 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 9, 2024-09-05T02:10:02.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.294394139, 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.117585109, 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.063827092, 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.093094565, 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3765, 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.161385086, 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000042864 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4, 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000374, 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000093 2024-09-05T02:10:02.426 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 286096 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 191035, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 92906, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 67709445, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 38298688, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 11, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 7.053679608, 2024-09-05T02:10:02.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 2.864982718, 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 5.769149667, 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.035844796, 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 92905, 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.631980875, 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000017566 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 277521, 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.032918506, 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000118 2024-09-05T02:10:02.428 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 597440, 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 110032 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 6503, 2024-09-05T02:10:02.429 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 3.698339786, 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000568712 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4009, 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.301396416, 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000075179 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-09-05T02:10:02.430 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 41159, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 27786, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 27786, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 283.982713271, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.010220352 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 13373, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 56, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 5, 2024-09-05T02:10:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 16, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 5, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 11, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 14, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 625, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 4, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 86245, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 43589, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 13373, 2024-09-05T02:10:02.432 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 1827, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 6, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 1800, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 63, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 179, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 28, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 1851, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-09-05T02:10:02.433 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 2690, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 59616945, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 74, 2024-09-05T02:10:02.434 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 4125, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 3533, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 103201, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 889, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 127, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 6496 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 2357, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 9649, 2024-09-05T02:10:02.435 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 7259, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 11625, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 1857, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 1769, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 1769, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:10:02.436 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 5, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 3677, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 63, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 5, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-09-05T02:10:02.437 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 31269, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 31269, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 37, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 37, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 4, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-09-05T02:10:02.438 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 34897738, 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3178, 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 7.524545537, 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002367698 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 25, 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 2047, 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 2022, 2024-09-05T02:10:02.439 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 36, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 2949, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 2913, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 132, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 5044, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 4912, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 1952, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 1952, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 78392, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 223520 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-09-05T02:10:02.440 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 50600, 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 2044, 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 41159, 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 133, 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 5063, 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 887, 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 188.720514053, 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.212762699 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-09-05T02:10:02.441 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3, 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.320541521, 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.106847173 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.442 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 24375, 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 11.883137070, 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000487513 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.443 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-09-05T02:10:02.444 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 211, 2024-09-05T02:10:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.635250548, 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003010666 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 103, 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.071857508, 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000697645 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 872, 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 80.164626637, 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.091931911 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.447 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-09-05T02:10:02.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.450 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1335, 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2.186785934, 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001638041 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.452 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 8233, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 8233, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 437, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-09-05T02:10:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:10:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-09-05T02:10:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 296464, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 109, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-09-05T02:10:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8952, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 168, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 211709, 2024-09-05T02:10:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2113, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 7628, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 31730350, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 33, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 7628, 2024-09-05T02:10:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 7628, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 20.490654621, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002686242 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 7628, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 8064 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 7628, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 224, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 7404, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-09-05T02:10:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 224, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 3402, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 150, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 5, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 3557, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-09-05T02:10:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-09-05T02:10:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-09-05T02:10:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 90, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 4, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 516, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 180 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-09-05T02:10:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 3689 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 5, 2024-09-05T02:10:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 3538, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 1769, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 207327, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 53933352, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 207327, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 207327, 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 53933352, 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-09-05T02:10:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 7628, 2024-09-05T02:10:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 31923171, 2024-09-05T02:10:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 7628, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 31923171, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 7628, 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 7628, 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 7628, 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 7628, 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:10:02.469 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 1769, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 192821, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 1769, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 224, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 192821, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.470 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x557a257b2960": { 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 31269, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 30703434, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 31269, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3178, 2024-09-05T02:10:02.471 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 30703434, 2024-09-05T02:10:02.472 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:02.472 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:02.472 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:02.472 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:02.472 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:02.472 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:02.472 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-05T02:10:02.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:02 smithi167 ceph-mon[35122]: pgmap v578: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-05T02:10:02.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:02 smithi139 ceph-mon[34262]: pgmap v578: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-05T02:10:02.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:02 smithi103 ceph-mon[30902]: pgmap v578: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-05T02:10:03.646 INFO:tasks.check_counter:Daemon mds.c mds.exported=63 2024-09-05T02:10:03.747 INFO:tasks.check_counter:Daemon mds.c mds.imported=28 2024-09-05T02:10:03.747 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=2357 2024-09-05T02:10:03.747 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=9649 2024-09-05T02:10:03.747 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=5 2024-09-05T02:10:03.747 DEBUG:tasks.check_counter:Getting stats from f 2024-09-05T02:10:03.747 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-05T02:10:03.998 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T02:10:04.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:04 smithi167 ceph-mon[35122]: pgmap v579: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-05T02:10:04.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:04 smithi139 ceph-mon[34262]: pgmap v579: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-05T02:10:04.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:04 smithi103 ceph-mon[30902]: pgmap v579: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1019, 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1021, 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 14664448, 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 324052, 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.089813155, 2024-09-05T02:10:04.658 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.025001247, 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.030058516, 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.026424132, 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1019, 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.033839491, 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000033208 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.659 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 432, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 556, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 16889813, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 422572, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.044612515, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.013647548, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.019278479, 2024-09-05T02:10:04.660 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.008435705, 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 555, 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.025877133, 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000046625 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 286720 2024-09-05T02:10:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 641, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 624, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 9078203, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 246133, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.061797165, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.022399243, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.030930965, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.010472705, 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:04.662 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 623, 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.022191432, 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000035620 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 577776, 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 104304 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-09-05T02:10:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3193, 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.106472216, 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000033345 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-09-05T02:10:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-09-05T02:10:04.665 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-09-05T02:10:04.665 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.665 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.665 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.665 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:04.665 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-09-05T02:10:04.666 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 2665, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 1373, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 1292, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 703, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 2, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-09-05T02:10:04.667 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-09-05T02:10:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 4, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 234, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-09-05T02:10:04.669 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-09-05T02:10:04.670 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:10:04.671 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-09-05T02:10:04.672 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 33812, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 3, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 32, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-09-05T02:10:04.673 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 41793212, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 43719403, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 43715102, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 33812 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 25, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 2837, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 2812, 2024-09-05T02:10:04.674 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 20, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 175, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 155, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 2665, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 6734, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 4069, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 53036, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 215328 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-09-05T02:10:04.675 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.676 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.677 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-09-05T02:10:04.679 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.680 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.681 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.682 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.683 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.684 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.685 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.686 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:04.687 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 3, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 2, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 1527, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.688 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:10:04.689 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:10:04.690 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 215191, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 66, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:10:04.691 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8952, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 168, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-09-05T02:10:04.692 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 1957909, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 11162, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 1892, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 26, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 4, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 1892, 2024-09-05T02:10:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1892, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.044982399, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000552316 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1892, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1893 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 1892, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 1892, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-09-05T02:10:04.694 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 1086, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 803, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-09-05T02:10:04.695 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 2, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-09-05T02:10:04.696 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-09-05T02:10:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 90, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 4, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 8, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 8, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-09-05T02:10:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-09-05T02:10:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2088, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 40431613, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2088, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2088, 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 40431613, 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1892, 2024-09-05T02:10:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 39525432, 2024-09-05T02:10:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1892, 2024-09-05T02:10:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 39525432, 2024-09-05T02:10:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1892, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1892, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1892, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1892, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-09-05T02:10:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55d5e8d02b40": { 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-09-05T02:10:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-05T02:10:05.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:05 smithi167 ceph-mon[35122]: pgmap v580: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 14 op/s 2024-09-05T02:10:05.453 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-05T02:10:05.453 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-05T02:10:05.453 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-05T02:10:05.453 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-05T02:10:05.453 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-05T02:10:05.453 DEBUG:tasks.check_counter:Getting stats from i 2024-09-05T02:10:05.453 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-05T02:10:05.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:05 smithi139 ceph-mon[34262]: pgmap v580: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 14 op/s 2024-09-05T02:10:05.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:05 smithi103 ceph-mon[30902]: pgmap v580: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 14 op/s 2024-09-05T02:10:05.704 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T02:10:06.204 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-05T02:10:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:10:06.204 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 70268, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 86986, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 15831987, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 91723206, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 11, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 3.560067123, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 1.452432474, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 2.330519674, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.155113754, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 86984, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.743443000, 2024-09-05T02:10:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000020043 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 144896, 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.022081723, 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000152 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 448806, 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 283554, 2024-09-05T02:10:06.206 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 144468187, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 147177824, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 12, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 15.503211174, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 6.354802379, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 12.198300224, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.128917058, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 283553, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 4.543813745, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000016024 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:06.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 708440, 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.087678354, 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000123 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 606224, 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 109472 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 7911, 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 8034, 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2286765, 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 53577347, 2024-09-05T02:10:06.208 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 14, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 7, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.615540512, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.255212871, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.133390216, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.188160330, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8033, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.298091653, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000037108 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4, 2024-09-05T02:10:06.209 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000277, 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000069 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 288160 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-09-05T02:10:06.210 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 21016, 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 25.102936249, 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001194467 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5957, 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.730797801, 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000122678 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 130658, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 81379, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 81379, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 919.638851381, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.011300689 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 49279, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 173, 2024-09-05T02:10:06.212 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 18, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 629, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 137, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 472, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 20, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 290, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 14413, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 2, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 2, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 6, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 200581, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 83148, 2024-09-05T02:10:06.213 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 48691, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 203, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 8, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 126, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 9865, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 329, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 1216, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:10:06.214 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 2690, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 59616945, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 4, 2024-09-05T02:10:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 22608, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 13043, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 245385, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 7492, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 210, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 34262 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 506, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 41, 2024-09-05T02:10:06.216 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 27, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 216, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 33896, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 10, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 2672, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 2662, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-09-05T02:10:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 5, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 8396, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 139, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 18, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 94489, 2024-09-05T02:10:06.218 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 94489, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 112, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 112, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 9, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 147158537, 2024-09-05T02:10:06.219 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 11410, 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 34.406254738, 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003015447 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 37, 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 7941, 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 7904, 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 99, 2024-09-05T02:10:06.220 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 546, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 447, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 495, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 19837, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 19342, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 2, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 7874, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 7872, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 91532, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 223520 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 141387, 2024-09-05T02:10:06.221 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 13951, 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 130658, 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 210, 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 27611, 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 7500, 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 152.978980424, 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.020397197 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 342, 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.570724010, 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.004592760 2024-09-05T02:10:06.222 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 63881, 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 520.927266336, 2024-09-05T02:10:06.223 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.008154651 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.224 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.225 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.226 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.226 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.226 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-09-05T02:10:06.226 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.226 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.226 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.226 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.226 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-09-05T02:10:06.226 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 61, 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 23.410970696, 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.383786404 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.227 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 489, 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 8.460411209, 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.017301454 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 129, 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.086771008, 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000672643 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 7516, 2024-09-05T02:10:06.228 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 209.020414451, 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.027810060 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.229 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 66, 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.255586315, 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.019024035 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.230 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.231 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.232 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1395, 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.927726932, 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001381883 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.233 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 16509, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 16509, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 443, 2024-09-05T02:10:06.234 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:10:06.235 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:10:06.236 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:10:06.237 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 311655, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 112, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-09-05T02:10:06.238 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8952, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 168, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 683246, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 5520, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-09-05T02:10:06.239 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 18423, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 146746627, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 28, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 18423, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 18423, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 48.214186558, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002617064 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 18423, 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 19161 2024-09-05T02:10:06.240 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 18423, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 337, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 18086, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 337, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 11747, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 233, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-09-05T02:10:06.241 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 18, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 5618, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-09-05T02:10:06.242 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-09-05T02:10:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 90, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 4, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-09-05T02:10:06.244 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 901, 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 277 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 8413 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.245 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 17, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 5, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 5572, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 2662, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 526981, 2024-09-05T02:10:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 115144043, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 526981, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 526981, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 115144043, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.247 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 18423, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 147036641, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 18423, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 147036641, 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:06.248 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 18423, 2024-09-05T02:10:06.249 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 18423, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 18423, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 18423, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2662, 2024-09-05T02:10:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 290014, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2662, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 337, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 290014, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:06.251 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x563d16a1c960": { 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 94489, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 142964233, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 94489, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 11410, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 142964233, 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:06.252 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:06.253 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:06.253 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:06.253 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:06.253 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:06.253 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-05T02:10:06.942 INFO:tasks.check_counter:Daemon mds.i mds.exported=126 2024-09-05T02:10:06.943 INFO:tasks.check_counter:Daemon mds.i mds.imported=329 2024-09-05T02:10:06.943 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=506 2024-09-05T02:10:06.943 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=41 2024-09-05T02:10:06.943 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=18 2024-09-05T02:10:06.943 DEBUG:tasks.check_counter:Getting stats from l 2024-09-05T02:10:06.943 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-05T02:10:07.189 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T02:10:07.694 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1192, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1192, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 51043651, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 378239, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.119358400, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.029347065, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.050705150, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.030660357, 2024-09-05T02:10:07.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1191, 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.041666752, 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000034984 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.696 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 465, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 595, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 50452285, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 436206, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.067245087, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.014270072, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.038420848, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.010345712, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 593, 2024-09-05T02:10:07.697 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.020469492, 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000034518 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 288592 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 484, 2024-09-05T02:10:07.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 467, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 42594934, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 197974, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 5, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.064344191, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.015333803, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.047982268, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.005257200, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 466, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.016749226, 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000035942 2024-09-05T02:10:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 578976, 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 107344 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.700 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3262, 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.071356381, 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000021875 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.701 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-09-05T02:10:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-09-05T02:10:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-09-05T02:10:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-09-05T02:10:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.702 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 3791, 2024-09-05T02:10:07.703 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 9, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 3782, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 3777, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 1, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-09-05T02:10:07.704 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-09-05T02:10:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-09-05T02:10:07.706 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-09-05T02:10:07.707 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-09-05T02:10:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-09-05T02:10:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-09-05T02:10:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-09-05T02:10:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-09-05T02:10:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-09-05T02:10:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-09-05T02:10:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-09-05T02:10:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-09-05T02:10:07.708 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-09-05T02:10:07.709 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 94489, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 112, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 9, 2024-09-05T02:10:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 147158537, 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 147157399, 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 94489 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-09-05T02:10:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 23, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 7840, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 7817, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 84, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 469, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 385, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 3791, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 16579, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 12788, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 58648, 2024-09-05T02:10:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 215328 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-09-05T02:10:07.713 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.714 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.715 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-09-05T02:10:07.716 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.717 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.719 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.721 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.723 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-09-05T02:10:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-09-05T02:10:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-09-05T02:10:07.725 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 1531, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-09-05T02:10:07.726 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T02:10:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-09-05T02:10:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T02:10:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T02:10:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T02:10:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T02:10:07.727 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T02:10:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-09-05T02:10:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T02:10:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T02:10:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-09-05T02:10:07.728 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-09-05T02:10:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-09-05T02:10:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-09-05T02:10:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T02:10:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T02:10:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T02:10:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T02:10:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T02:10:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-09-05T02:10:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 239448, 2024-09-05T02:10:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 79, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8952, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 168, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-09-05T02:10:07.731 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 2836122, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 19853, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 1937, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2024-09-05T02:10:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 1937, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1937, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.308977933, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000675775 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1937, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1938 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 1937, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 1937, 2024-09-05T02:10:07.733 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 1083, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 852, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-09-05T02:10:07.734 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-09-05T02:10:07.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T02:10:07.738 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-09-05T02:10:07.738 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-09-05T02:10:07.738 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-09-05T02:10:07.738 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-09-05T02:10:07.739 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-09-05T02:10:07.739 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-09-05T02:10:07.739 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-09-05T02:10:07.739 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-09-05T02:10:07.739 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-09-05T02:10:07.739 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-09-05T02:10:07.739 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-09-05T02:10:07.739 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-09-05T02:10:07.740 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-09-05T02:10:07.740 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-09-05T02:10:07.740 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-09-05T02:10:07.740 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-09-05T02:10:07.740 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-09-05T02:10:07.740 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-09-05T02:10:07.740 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-09-05T02:10:07.740 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 90, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 4, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 11, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 11, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-09-05T02:10:07.741 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-09-05T02:10:07.742 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:07.743 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2137, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 143885395, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2137, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2137, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 143885395, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.744 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1937, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 142969850, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1937, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 142969850, 2024-09-05T02:10:07.745 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:07.746 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1937, 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1937, 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1937, 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1937, 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-09-05T02:10:07.747 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:07.748 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x56284fbb8b40": { 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-09-05T02:10:07.749 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-05T02:10:07.750 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-05T02:10:08.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:08 smithi167 ceph-mon[35122]: pgmap v581: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-09-05T02:10:08.540 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-05T02:10:08.540 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-05T02:10:08.540 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-05T02:10:08.540 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-05T02:10:08.541 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-05T02:10:08.541 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-05T02:10:08.553 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-05T02:10:08.553 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T02:10:08.553 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T02:10:08.576 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T02:10:08.576 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T02:10:08.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T02:10:08.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:08 smithi139 ceph-mon[34262]: pgmap v581: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-09-05T02:10:08.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:08 smithi103 ceph-mon[30902]: pgmap v581: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-09-05T02:10:08.907 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:09.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.579+0000 7f4e5e556640 1 -- 172.21.15.103:0/475232188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102570 msgr2=0x7f4e581029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:09.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.579+0000 7f4e5e556640 1 --2- 172.21.15.103:0/475232188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102570 0x7f4e581029b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009900 tx=0x7f4e4c02a480 comp rx=0 tx=0).stop 2024-09-05T02:10:09.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.579+0000 7f4e5e556640 1 -- 172.21.15.103:0/475232188 shutdown_connections 2024-09-05T02:10:09.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.579+0000 7f4e5e556640 1 --2- 172.21.15.103:0/475232188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e58102ef0 0x7f4e5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.579+0000 7f4e5e556640 1 --2- 172.21.15.103:0/475232188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102570 0x7f4e581029b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.579+0000 7f4e5e556640 1 --2- 172.21.15.103:0/475232188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e58100e20 0x7f4e58101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.579+0000 7f4e5e556640 1 -- 172.21.15.103:0/475232188 >> 172.21.15.103:0/475232188 conn(0x7f4e580fc7d0 msgr2=0x7f4e580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:09.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.579+0000 7f4e5e556640 1 -- 172.21.15.103:0/475232188 shutdown_connections 2024-09-05T02:10:09.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.579+0000 7f4e5e556640 1 -- 172.21.15.103:0/475232188 wait complete. 2024-09-05T02:10:09.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.580+0000 7f4e5e556640 1 Processor -- start 2024-09-05T02:10:09.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.580+0000 7f4e5e556640 1 -- start start 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.580+0000 7f4e5e556640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e58100e20 0x7f4e580fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.580+0000 7f4e5e556640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e58102570 0x7f4e580fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.580+0000 7f4e5e556640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102ef0 0x7f4e580fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.580+0000 7f4e5e556640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58110e50 con 0x7f4e58102ef0 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.580+0000 7f4e5e556640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58110cd0 con 0x7f4e58102570 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.580+0000 7f4e5e556640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58110fd0 con 0x7f4e58100e20 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.580+0000 7f4e5caf3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102ef0 0x7f4e580fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5caf3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102ef0 0x7f4e580fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59936/0 (socket says 172.21.15.103:59936) 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5caf3640 1 -- 172.21.15.103:0/2045942239 learned_addr learned my addr 172.21.15.103:0/2045942239 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e577fe640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e58102570 0x7f4e580fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:09.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e57fff640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e58100e20 0x7f4e580fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:09.852 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2384335573 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:09.852 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2384335573 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:09.852 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2155492495 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:09.852 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/396485172 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:09.852 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1467689006 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:09.853 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3874104813 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:09.853 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1573405851 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:09.853 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2132077607 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:09.853 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3892241075 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:09.853 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2132077607 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:09.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5caf3640 1 -- 172.21.15.103:0/2045942239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e58100e20 msgr2=0x7f4e580fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:09.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5caf3640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e58100e20 0x7f4e580fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5caf3640 1 -- 172.21.15.103:0/2045942239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e58102570 msgr2=0x7f4e580fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5caf3640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e58102570 0x7f4e580fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5caf3640 1 -- 172.21.15.103:0/2045942239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e581a69a0 con 0x7f4e58102ef0 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5caf3640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102ef0 0x7f4e580fd700 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000eb10 tx=0x7f4e5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e557fa640 1 -- 172.21.15.103:0/2045942239 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50004b20 con 0x7f4e58102ef0 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e581a6c90 con 0x7f4e58102ef0 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e557fa640 1 -- 172.21.15.103:0/2045942239 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e50004cc0 con 0x7f4e58102ef0 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e557fa640 1 -- 172.21.15.103:0/2045942239 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50010690 con 0x7f4e58102ef0 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.581+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581a71a0 con 0x7f4e58102ef0 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.582+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e58101200 con 0x7f4e58102ef0 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.583+0000 7f4e557fa640 1 -- 172.21.15.103:0/2045942239 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4e50005020 con 0x7f4e58102ef0 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.584+0000 7f4e557fa640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4e400a2b80 0x7f4e400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:09.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.584+0000 7f4e57fff640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4e400a2b80 0x7f4e400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.584+0000 7f4e557fa640 1 -- 172.21.15.103:0/2045942239 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4e500cfa30 con 0x7f4e58102ef0 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.585+0000 7f4e57fff640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4e400a2b80 0x7f4e400a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009890 tx=0x7f4e48008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.588+0000 7f4e557fa640 1 -- 172.21.15.103:0/2045942239 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e5009c5e0 con 0x7f4e58102ef0 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.767+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4e5810a0a0 con 0x7f4e58102ef0 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.767+0000 7f4e557fa640 1 -- 172.21.15.103:0/2045942239 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+601 (secure 0 0 0) 0x7f4e5009bee0 con 0x7f4e58102ef0 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4e400a2b80 msgr2=0x7f4e400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4e400a2b80 0x7f4e400a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009890 tx=0x7f4e48008040 comp rx=0 tx=0).stop 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102ef0 msgr2=0x7f4e580fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102ef0 0x7f4e580fd700 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000eb10 tx=0x7f4e5000c2d0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 shutdown_connections 2024-09-05T02:10:09.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e58102ef0 0x7f4e580fd700 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e58102570 0x7f4e580fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f4e400a2b80 0x7f4e400a5040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 --2- 172.21.15.103:0/2045942239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e58100e20 0x7f4e580fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:09.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 >> 172.21.15.103:0/2045942239 conn(0x7f4e580fc7d0 msgr2=0x7f4e5806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:09.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 shutdown_connections 2024-09-05T02:10:09.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:09.770+0000 7f4e5e556640 1 -- 172.21.15.103:0/2045942239 wait complete. 2024-09-05T02:10:10.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:10 smithi167 ceph-mon[35122]: pgmap v582: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T02:10:10.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:10 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2045942239' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:10:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:10 smithi139 ceph-mon[34262]: pgmap v582: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T02:10:10.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:10 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2045942239' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:10:10.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:10 smithi103 ceph-mon[30902]: pgmap v582: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T02:10:10.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:10 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2045942239' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:10:10.676 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-05T02:10:10.676 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T02:10:10.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T02:10:10.982 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:11.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 -- 172.21.15.103:0/1808971546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d0101df0 msgr2=0x7fe6d01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/1808971546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d0101df0 0x7fe6d01021d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009900 tx=0x7fe6c002a410 comp rx=0 tx=0).stop 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 -- 172.21.15.103:0/1808971546 shutdown_connections 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/1808971546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d0106f20 0x7fe6d010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/1808971546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6d01027a0 0x7fe6d01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/1808971546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d0101df0 0x7fe6d01021d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 -- 172.21.15.103:0/1808971546 >> 172.21.15.103:0/1808971546 conn(0x7fe6d00fd8b0 msgr2=0x7fe6d00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 -- 172.21.15.103:0/1808971546 shutdown_connections 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 -- 172.21.15.103:0/1808971546 wait complete. 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.477+0000 7fe6d6a86640 1 Processor -- start 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d6a86640 1 -- start start 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d6a86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6d0101df0 0x7fe6d01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:11.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d6a86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d01027a0 0x7fe6d0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d6a86640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d0106f20 0x7fe6d019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d6a86640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0111e30 con 0x7fe6d01027a0 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d6a86640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0111cb0 con 0x7fe6d0106f20 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d6a86640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0111fb0 con 0x7fe6d0101df0 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d01027a0 0x7fe6d0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d01027a0 0x7fe6d0198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59956/0 (socket says 172.21.15.103:59956) 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6cffff640 1 -- 172.21.15.103:0/2745563999 learned_addr learned my addr 172.21.15.103:0/2745563999 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d4822640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6d0101df0 0x7fe6d01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d5023640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d0106f20 0x7fe6d019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6cffff640 1 -- 172.21.15.103:0/2745563999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6d0101df0 msgr2=0x7fe6d01982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6cffff640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6d0101df0 0x7fe6d01982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:11.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6cffff640 1 -- 172.21.15.103:0/2745563999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d0106f20 msgr2=0x7fe6d019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6cffff640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d0106f20 0x7fe6d019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6cffff640 1 -- 172.21.15.103:0/2745563999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d019d280 con 0x7fe6d01027a0 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6d5023640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d0106f20 0x7fe6d019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.478+0000 7fe6cffff640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d01027a0 0x7fe6d0198810 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d770 tx=0x7fe6c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.479+0000 7fe6cdffb640 1 -- 172.21.15.103:0/2745563999 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4017070 con 0x7fe6d01027a0 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.479+0000 7fe6cdffb640 1 -- 172.21.15.103:0/2745563999 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6c40054d0 con 0x7fe6d01027a0 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.479+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d019d570 con 0x7fe6d01027a0 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.479+0000 7fe6cdffb640 1 -- 172.21.15.103:0/2745563999 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4004cf0 con 0x7fe6d01027a0 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.479+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d01a4e50 con 0x7fe6d01027a0 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.480+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d01021d0 con 0x7fe6d01027a0 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.481+0000 7fe6cdffb640 1 -- 172.21.15.103:0/2745563999 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe6c4004e90 con 0x7fe6d01027a0 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.481+0000 7fe6cdffb640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6b80a2b80 0x7fe6b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:11.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.481+0000 7fe6d4822640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6b80a2b80 0x7fe6b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:11.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.482+0000 7fe6cdffb640 1 -- 172.21.15.103:0/2745563999 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe6c40cee60 con 0x7fe6d01027a0 2024-09-05T02:10:11.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.482+0000 7fe6d4822640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6b80a2b80 0x7fe6b80a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0031440 tx=0x7fe6c0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:11.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.486+0000 7fe6cdffb640 1 -- 172.21.15.103:0/2745563999 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6c409ba10 con 0x7fe6d01027a0 2024-09-05T02:10:11.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.665+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe6d010a100 con 0x7fe6d01027a0 2024-09-05T02:10:11.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.665+0000 7fe6cdffb640 1 -- 172.21.15.103:0/2745563999 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+601 (secure 0 0 0) 0x7fe6c409b310 con 0x7fe6d01027a0 2024-09-05T02:10:11.666 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2384335573 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:11.666 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2384335573 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:11.667 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2155492495 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:11.667 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/396485172 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:11.667 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1467689006 2024-09-06T01:53:25.001822+0000 2024-09-05T02:10:11.667 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3874104813 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:11.667 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1573405851 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:11.667 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2132077607 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:11.667 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3892241075 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:11.667 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2132077607 2024-09-06T01:53:03.000202+0000 2024-09-05T02:10:11.667 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6b80a2b80 msgr2=0x7fe6b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6b80a2b80 0x7fe6b80a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0031440 tx=0x7fe6c0005b50 comp rx=0 tx=0).stop 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d01027a0 msgr2=0x7fe6d0198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d01027a0 0x7fe6d0198810 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d770 tx=0x7fe6c400dc30 comp rx=0 tx=0).stop 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 shutdown_connections 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d0106f20 0x7fe6d019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d01027a0 0x7fe6d0198810 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe6b80a2b80 0x7fe6b80a5040 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 --2- 172.21.15.103:0/2745563999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6d0101df0 0x7fe6d01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 >> 172.21.15.103:0/2745563999 conn(0x7fe6d00fd8b0 msgr2=0x7fe6d00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 shutdown_connections 2024-09-05T02:10:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:11.668+0000 7fe6d6a86640 1 -- 172.21.15.103:0/2745563999 wait complete. 2024-09-05T02:10:12.338 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi103.front.sepia.ceph.com... 2024-09-05T02:10:12.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:10:12.339 DEBUG:teuthology.orchestra.run.smithi103:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-05T02:10:12.365 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi103.stderr:ceph-fuse[82453]: fuse finished with error 0 and tester_r 0 2024-09-05T02:10:12.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T02:10:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:12 smithi167 ceph-mon[35122]: pgmap v583: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2024-09-05T02:10:12.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:12 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2745563999' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:10:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:12 smithi139 ceph-mon[34262]: pgmap v583: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2024-09-05T02:10:12.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:12 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2745563999' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:10:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:12 smithi103 ceph-mon[30902]: pgmap v583: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2024-09-05T02:10:12.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:12 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2745563999' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T02:10:14.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:14 smithi167 ceph-mon[35122]: pgmap v584: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-05T02:10:14.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:14 smithi139 ceph-mon[34262]: pgmap v584: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-05T02:10:14.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:14 smithi103 ceph-mon[30902]: pgmap v584: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-05T02:10:16.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:16 smithi167 ceph-mon[35122]: pgmap v585: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-05T02:10:16.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:16 smithi139 ceph-mon[34262]: pgmap v585: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-05T02:10:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:16 smithi103 ceph-mon[30902]: pgmap v585: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-05T02:10:18.468 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-09-05T02:10:18.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:10:18.469 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-05T02:10:18.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:10:18.489 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-05T02:10:18.548 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-05T02:10:18.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:10:18.549 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T02:10:18.613 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:18 smithi103 ceph-mon[30902]: pgmap v586: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-05T02:10:18.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T02:10:18.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-05T02:10:18.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:18 smithi167 ceph-mon[35122]: pgmap v586: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-05T02:10:18.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:18 smithi139 ceph-mon[34262]: pgmap v586: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-05T02:10:18.708 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-05T02:10:18.719 INFO:tasks.cephadm:Teardown begin 2024-09-05T02:10:18.719 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-05T02:10:18.719 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:10:18.973 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:19.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:19 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:10:19.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.553+0000 7fe64a159640 1 -- 172.21.15.103:0/475094782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe644102d40 msgr2=0x7fe644103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.553+0000 7fe64a159640 1 --2- 172.21.15.103:0/475094782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe644102d40 0x7fe644103120 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe634009900 tx=0x7fe63402a480 comp rx=0 tx=0).stop 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.553+0000 7fe64a159640 1 -- 172.21.15.103:0/475094782 shutdown_connections 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.553+0000 7fe64a159640 1 --2- 172.21.15.103:0/475094782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe64410c170 0x7fe64410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.553+0000 7fe64a159640 1 --2- 172.21.15.103:0/475094782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe644103660 0x7fe64410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.553+0000 7fe64a159640 1 --2- 172.21.15.103:0/475094782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe644102d40 0x7fe644103120 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.553+0000 7fe64a159640 1 -- 172.21.15.103:0/475094782 >> 172.21.15.103:0/475094782 conn(0x7fe6440fc7a0 msgr2=0x7fe6440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.553+0000 7fe64a159640 1 -- 172.21.15.103:0/475094782 shutdown_connections 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.553+0000 7fe64a159640 1 -- 172.21.15.103:0/475094782 wait complete. 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe64a159640 1 Processor -- start 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe64a159640 1 -- start start 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe64a159640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe644102d40 0x7fe6441009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe64a159640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe644103660 0x7fe644100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe64a159640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe64410c170 0x7fe644101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe64a159640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe644111e20 con 0x7fe64410c170 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe64a159640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe644111ca0 con 0x7fe644103660 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe64a159640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe644111fa0 con 0x7fe644102d40 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe643fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe64410c170 0x7fe644101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe643fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe64410c170 0x7fe644101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35626/0 (socket says 172.21.15.103:35626) 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe643fff640 1 -- 172.21.15.103:0/532177720 learned_addr learned my addr 172.21.15.103:0/532177720 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe6437fe640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe644102d40 0x7fe6441009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.554+0000 7fe642ffd640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe644103660 0x7fe644100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe643fff640 1 -- 172.21.15.103:0/532177720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe644102d40 msgr2=0x7fe6441009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe643fff640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe644102d40 0x7fe6441009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe643fff640 1 -- 172.21.15.103:0/532177720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe644103660 msgr2=0x7fe644100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe643fff640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe644103660 0x7fe644100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe643fff640 1 -- 172.21.15.103:0/532177720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6441a68f0 con 0x7fe64410c170 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe643fff640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe64410c170 0x7fe644101420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00efd0 tx=0x7fe63c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe640ff9640 1 -- 172.21.15.103:0/532177720 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c003b70 con 0x7fe64410c170 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe640ff9640 1 -- 172.21.15.103:0/532177720 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe63c003d10 con 0x7fe64410c170 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6441a6be0 con 0x7fe64410c170 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe640ff9640 1 -- 172.21.15.103:0/532177720 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c010980 con 0x7fe64410c170 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.555+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6441a70d0 con 0x7fe64410c170 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.556+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe644103120 con 0x7fe64410c170 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.557+0000 7fe640ff9640 1 -- 172.21.15.103:0/532177720 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe63c004b40 con 0x7fe64410c170 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.558+0000 7fe640ff9640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe62c0a2b80 0x7fe62c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.558+0000 7fe6437fe640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe62c0a2b80 0x7fe62c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.558+0000 7fe640ff9640 1 -- 172.21.15.103:0/532177720 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe63c0cfb20 con 0x7fe64410c170 2024-09-05T02:10:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.559+0000 7fe6437fe640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe62c0a2b80 0x7fe62c0a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe634031440 tx=0x7fe634005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.561+0000 7fe640ff9640 1 -- 172.21.15.103:0/532177720 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe63c09c6d0 con 0x7fe64410c170 2024-09-05T02:10:19.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:19 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:10:19.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:19 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T02:10:19.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.773+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe644101ed0 con 0x7fe64410c170 2024-09-05T02:10:19.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.774+0000 7fe640ff9640 1 -- 172.21.15.103:0/532177720 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+12033 (secure 0 0 0) 0x7fe644101ed0 con 0x7fe64410c170 2024-09-05T02:10:19.776 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:10:19.777 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":42,"btime":"2024-09-05T02:09:29:302685+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:09:28.699103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 42 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe62c0a2b80 msgr2=0x7fe62c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe62c0a2b80 0x7fe62c0a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe634031440 tx=0x7fe634005b50 comp rx=0 tx=0).stop 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe64410c170 msgr2=0x7fe644101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe64410c170 0x7fe644101420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00efd0 tx=0x7fe63c00c6a0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 shutdown_connections 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe64410c170 0x7fe644101420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe644103660 0x7fe644100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe62c0a2b80 0x7fe62c0a5040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 --2- 172.21.15.103:0/532177720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe644102d40 0x7fe6441009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:19.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 >> 172.21.15.103:0/532177720 conn(0x7fe6440fc7a0 msgr2=0x7fe6440fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:19.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 shutdown_connections 2024-09-05T02:10:19.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:19.777+0000 7fe64a159640 1 -- 172.21.15.103:0/532177720 wait complete. 2024-09-05T02:10:20.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:20 smithi103 ceph-mon[30902]: pgmap v587: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T02:10:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:10:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:10:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:20 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:10:20.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:20 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/532177720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:10:20.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T02:10:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:20 smithi167 ceph-mon[35122]: pgmap v587: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T02:10:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:10:20.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:10:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:20 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:10:20.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:20 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/532177720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:10:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:20 smithi139 ceph-mon[34262]: pgmap v587: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T02:10:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T02:10:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T02:10:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:20 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' 2024-09-05T02:10:20.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:20 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/532177720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:10:20.700 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:21.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.203+0000 7efcf8c73640 1 -- 172.21.15.103:0/3327270326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 msgr2=0x7efcf41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:21.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.203+0000 7efcf8c73640 1 --2- 172.21.15.103:0/3327270326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 0x7efcf41067e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efce8009900 tx=0x7efce802a410 comp rx=0 tx=0).stop 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.203+0000 7efcf8c73640 1 -- 172.21.15.103:0/3327270326 shutdown_connections 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.203+0000 7efcf8c73640 1 --2- 172.21.15.103:0/3327270326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcf4106f10 0x7efcf410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.203+0000 7efcf8c73640 1 --2- 172.21.15.103:0/3327270326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 0x7efcf41067e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.203+0000 7efcf8c73640 1 --2- 172.21.15.103:0/3327270326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efcf4101de0 0x7efcf41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.203+0000 7efcf8c73640 1 -- 172.21.15.103:0/3327270326 >> 172.21.15.103:0/3327270326 conn(0x7efcf40fd8c0 msgr2=0x7efcf40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.203+0000 7efcf8c73640 1 -- 172.21.15.103:0/3327270326 shutdown_connections 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.203+0000 7efcf8c73640 1 -- 172.21.15.103:0/3327270326 wait complete. 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.204+0000 7efcf8c73640 1 Processor -- start 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.204+0000 7efcf8c73640 1 -- start start 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.204+0000 7efcf8c73640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efcf4101de0 0x7efcf41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.204+0000 7efcf8c73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 0x7efcf4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.204+0000 7efcf8c73640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcf4106f10 0x7efcf419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:21.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.204+0000 7efcf8c73640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4111e40 con 0x7efcf4102790 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.204+0000 7efcf8c73640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4111cc0 con 0x7efcf4106f10 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.204+0000 7efcf8c73640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efcf4111fc0 con 0x7efcf4101de0 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.204+0000 7efcf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 0x7efcf4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 0x7efcf4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35644/0 (socket says 172.21.15.103:35644) 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf1d9b640 1 -- 172.21.15.103:0/897575134 learned_addr learned my addr 172.21.15.103:0/897575134 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf259c640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efcf4101de0 0x7efcf41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf2d9d640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcf4106f10 0x7efcf419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf1d9b640 1 -- 172.21.15.103:0/897575134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efcf4101de0 msgr2=0x7efcf41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf1d9b640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efcf4101de0 0x7efcf41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf1d9b640 1 -- 172.21.15.103:0/897575134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcf4106f10 msgr2=0x7efcf419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf1d9b640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcf4106f10 0x7efcf419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf1d9b640 1 -- 172.21.15.103:0/897575134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf419d370 con 0x7efcf4102790 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf1d9b640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 0x7efcf4198900 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efce8006010 tx=0x7efce8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:21.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efce37fe640 1 -- 172.21.15.103:0/897575134 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce803f030 con 0x7efcf4102790 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efce37fe640 1 -- 172.21.15.103:0/897575134 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efce80035f0 con 0x7efcf4102790 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efce37fe640 1 -- 172.21.15.103:0/897575134 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8035070 con 0x7efcf4102790 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf419d600 con 0x7efcf4102790 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.205+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf41a4f40 con 0x7efcf4102790 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.206+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf41032c0 con 0x7efcf4102790 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.208+0000 7efce37fe640 1 -- 172.21.15.103:0/897575134 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efce8003b10 con 0x7efcf4102790 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.208+0000 7efce37fe640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efcdc0a2b80 0x7efcdc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.208+0000 7efcf259c640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efcdc0a2b80 0x7efcdc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.208+0000 7efce37fe640 1 -- 172.21.15.103:0/897575134 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efce80f0230 con 0x7efcf4102790 2024-09-05T02:10:21.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.209+0000 7efcf259c640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efcdc0a2b80 0x7efcdc0a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efce40058c0 tx=0x7efce400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:21.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.211+0000 7efce37fe640 1 -- 172.21.15.103:0/897575134 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efce80bcde0 con 0x7efcf4102790 2024-09-05T02:10:21.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.419+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcf410a150 con 0x7efcf4102790 2024-09-05T02:10:21.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.420+0000 7efce37fe640 1 -- 172.21.15.103:0/897575134 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+12033 (secure 0 0 0) 0x7efce80bc6e0 con 0x7efcf4102790 2024-09-05T02:10:21.421 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:10:21.421 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":42,"btime":"2024-09-05T02:09:29:302685+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/726289988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":726289988},{"type":"v1","addr":"172.21.15.103:6835","nonce":726289988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2025990895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2025990895},{"type":"v1","addr":"172.21.15.103:6837","nonce":2025990895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T02:00:15.086875+0000","modified":"2024-09-05T02:09:28.699103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24466,"mds_2":24464,"mds_3":24457,"mds_4":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1603518036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1603518036},{"type":"v1","addr":"172.21.15.103:6839","nonce":1603518036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/426074448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":426074448},{"type":"v1","addr":"172.21.15.103:6841","nonce":426074448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6833/4042514104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4042514104},{"type":"v1","addr":"172.21.15.139:6833","nonce":4042514104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2095779466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2095779466},{"type":"v1","addr":"172.21.15.167:6833","nonce":2095779466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/1110459707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1110459707},{"type":"v1","addr":"172.21.15.167:6835","nonce":1110459707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.139:6835/3204357322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3204357322},{"type":"v1","addr":"172.21.15.139:6835","nonce":3204357322}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3468047781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3468047781},{"type":"v1","addr":"172.21.15.167:6837","nonce":3468047781}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.139:6837/2336570561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2336570561},{"type":"v1","addr":"172.21.15.139:6837","nonce":2336570561}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.167:6839/1858062899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1858062899},{"type":"v1","addr":"172.21.15.167:6839","nonce":1858062899}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6839/3082363979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":3082363979},{"type":"v1","addr":"172.21.15.139:6839","nonce":3082363979}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24466,24464,24457,24449]},"id":1}]} 2024-09-05T02:10:21.422 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 42 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efcdc0a2b80 msgr2=0x7efcdc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efcdc0a2b80 0x7efcdc0a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efce40058c0 tx=0x7efce400a4c0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 msgr2=0x7efcf4198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 0x7efcf4198900 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efce8006010 tx=0x7efce8004270 comp rx=0 tx=0).stop 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 shutdown_connections 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcf4106f10 0x7efcf419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf4102790 0x7efcf4198900 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7efcdc0a2b80 0x7efcdc0a5040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 --2- 172.21.15.103:0/897575134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efcf4101de0 0x7efcf41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 >> 172.21.15.103:0/897575134 conn(0x7efcf40fd8c0 msgr2=0x7efcf40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 shutdown_connections 2024-09-05T02:10:21.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:21.424+0000 7efcf8c73640 1 -- 172.21.15.103:0/897575134 wait complete. 2024-09-05T02:10:22.161 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T02:10:22.398 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:22 smithi103 ceph-mon[30902]: pgmap v588: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-05T02:10:22.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:22 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/897575134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:10:22.409 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:22 smithi167 ceph-mon[35122]: pgmap v588: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-05T02:10:22.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:22 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/897575134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:10:22.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:22 smithi139 ceph-mon[34262]: pgmap v588: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-05T02:10:22.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:22 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/897575134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.875+0000 7fd098bf7640 1 -- 172.21.15.103:0/1062158583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 msgr2=0x7fd0941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.875+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1062158583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 0x7fd0941021c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd084009900 tx=0x7fd08402a410 comp rx=0 tx=0).stop 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.875+0000 7fd098bf7640 1 -- 172.21.15.103:0/1062158583 shutdown_connections 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.875+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1062158583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd094106f10 0x7fd09410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.875+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1062158583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094102790 0x7fd0941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.875+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1062158583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 0x7fd0941021c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.875+0000 7fd098bf7640 1 -- 172.21.15.103:0/1062158583 >> 172.21.15.103:0/1062158583 conn(0x7fd0940fd8c0 msgr2=0x7fd0940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.875+0000 7fd098bf7640 1 -- 172.21.15.103:0/1062158583 shutdown_connections 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.875+0000 7fd098bf7640 1 -- 172.21.15.103:0/1062158583 wait complete. 2024-09-05T02:10:22.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd098bf7640 1 Processor -- start 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd098bf7640 1 -- start start 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd098bf7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 0x7fd0940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd098bf7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094102790 0x7fd094074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd098bf7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd094106f10 0x7fd094074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd098bf7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd094111e40 con 0x7fd094101de0 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd098bf7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd094111cc0 con 0x7fd094106f10 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd098bf7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd094111fc0 con 0x7fd094102790 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd09259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 0x7fd0940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd09259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 0x7fd0940744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35674/0 (socket says 172.21.15.103:35674) 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd09259c640 1 -- 172.21.15.103:0/1661898354 learned_addr learned my addr 172.21.15.103:0/1661898354 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:22.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd092d9d640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd094106f10 0x7fd094074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd091d9b640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094102790 0x7fd094074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd09259c640 1 -- 172.21.15.103:0/1661898354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094102790 msgr2=0x7fd094074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd09259c640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094102790 0x7fd094074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd09259c640 1 -- 172.21.15.103:0/1661898354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd094106f10 msgr2=0x7fd094074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd09259c640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd094106f10 0x7fd094074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd09259c640 1 -- 172.21.15.103:0/1661898354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0941a68e0 con 0x7fd094101de0 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd091d9b640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094102790 0x7fd094074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.876+0000 7fd09259c640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 0x7fd0940744f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd084006010 tx=0x7fd084004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.877+0000 7fd0837fe640 1 -- 172.21.15.103:0/1661898354 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08403f030 con 0x7fd094101de0 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.877+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0941a6b70 con 0x7fd094101de0 2024-09-05T02:10:22.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.877+0000 7fd0837fe640 1 -- 172.21.15.103:0/1661898354 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0840035f0 con 0x7fd094101de0 2024-09-05T02:10:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.877+0000 7fd0837fe640 1 -- 172.21.15.103:0/1661898354 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084035070 con 0x7fd094101de0 2024-09-05T02:10:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.877+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0941a7050 con 0x7fd094101de0 2024-09-05T02:10:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.878+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0941021c0 con 0x7fd094101de0 2024-09-05T02:10:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.879+0000 7fd0837fe640 1 -- 172.21.15.103:0/1661898354 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd084003b10 con 0x7fd094101de0 2024-09-05T02:10:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.879+0000 7fd0837fe640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd07c0a2b80 0x7fd07c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.880+0000 7fd091d9b640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd07c0a2b80 0x7fd07c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.880+0000 7fd0837fe640 1 -- 172.21.15.103:0/1661898354 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd0840f0020 con 0x7fd094101de0 2024-09-05T02:10:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.880+0000 7fd091d9b640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd07c0a2b80 0x7fd07c0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd088006fd0 tx=0x7fd088008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:22.884+0000 7fd0837fe640 1 -- 172.21.15.103:0/1661898354 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0840bcbd0 con 0x7fd094101de0 2024-09-05T02:10:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.052+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd094075a20 con 0x7fd094101de0 2024-09-05T02:10:23.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.053+0000 7fd0837fe640 1 -- 172.21.15.103:0/1661898354 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24926 (secure 0 0 0) 0x7fd0840bc4d0 con 0x7fd094101de0 2024-09-05T02:10:23.054 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-05T02:10:23.054 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":90,"fsid":"695e8e42-6b29-11ef-bcd6-c7b262605968","created":"2024-09-05T01:52:29.045755+0000","modified":"2024-09-05T02:02:27.192641+0000","last_up_change":"2024-09-05T01:59:19.839171+0000","last_in_change":"2024-09-05T01:58:56.543888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:55:35.312049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T02:00:11.283498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T02:00:13.180482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T02:00:19.622395+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6fe0c785-c54d-4779-8255-8d88e39aa32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6803","nonce":2389557969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6805","nonce":2389557969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6809","nonce":2389557969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2389557969},{"type":"v1","addr":"172.21.15.103:6807","nonce":2389557969}]},"public_addr":"172.21.15.103:6803/2389557969","cluster_addr":"172.21.15.103:6805/2389557969","heartbeat_back_addr":"172.21.15.103:6809/2389557969","heartbeat_front_addr":"172.21.15.103:6807/2389557969","state":["exists","up"]},{"osd":1,"uuid":"e0815a57-5a14-4d03-9045-36fd3649ab31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6801","nonce":491231014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6803","nonce":491231014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6807","nonce":491231014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":491231014},{"type":"v1","addr":"172.21.15.139:6805","nonce":491231014}]},"public_addr":"172.21.15.139:6801/491231014","cluster_addr":"172.21.15.139:6803/491231014","heartbeat_back_addr":"172.21.15.139:6807/491231014","heartbeat_front_addr":"172.21.15.139:6805/491231014","state":["exists","up"]},{"osd":2,"uuid":"13eeb8f3-8b22-415b-abd9-d8d054bc1093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6801","nonce":945108637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6803","nonce":945108637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6807","nonce":945108637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":945108637},{"type":"v1","addr":"172.21.15.167:6805","nonce":945108637}]},"public_addr":"172.21.15.167:6801/945108637","cluster_addr":"172.21.15.167:6803/945108637","heartbeat_back_addr":"172.21.15.167:6807/945108637","heartbeat_front_addr":"172.21.15.167:6805/945108637","state":["exists","up"]},{"osd":3,"uuid":"80e9c6a8-bf1d-45a1-9e94-6abba2659c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6811","nonce":845945713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6813","nonce":845945713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6817","nonce":845945713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":845945713},{"type":"v1","addr":"172.21.15.103:6815","nonce":845945713}]},"public_addr":"172.21.15.103:6811/845945713","cluster_addr":"172.21.15.103:6813/845945713","heartbeat_back_addr":"172.21.15.103:6817/845945713","heartbeat_front_addr":"172.21.15.103:6815/845945713","state":["exists","up"]},{"osd":4,"uuid":"ed03c495-8ced-4b29-af59-3fec3dab74fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6809","nonce":2565711850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6811","nonce":2565711850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6815","nonce":2565711850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2565711850},{"type":"v1","addr":"172.21.15.139:6813","nonce":2565711850}]},"public_addr":"172.21.15.139:6809/2565711850","cluster_addr":"172.21.15.139:6811/2565711850","heartbeat_back_addr":"172.21.15.139:6815/2565711850","heartbeat_front_addr":"172.21.15.139:6813/2565711850","state":["exists","up"]},{"osd":5,"uuid":"37eca6d4-b597-41fc-af54-5db67c9a4d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6809","nonce":4050841663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6811","nonce":4050841663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6815","nonce":4050841663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":4050841663},{"type":"v1","addr":"172.21.15.167:6813","nonce":4050841663}]},"public_addr":"172.21.15.167:6809/4050841663","cluster_addr":"172.21.15.167:6811/4050841663","heartbeat_back_addr":"172.21.15.167:6815/4050841663","heartbeat_front_addr":"172.21.15.167:6813/4050841663","state":["exists","up"]},{"osd":6,"uuid":"82343e71-6c0c-4ea3-b050-23b2bcae9e50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6819","nonce":2808952554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6821","nonce":2808952554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6825","nonce":2808952554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":2808952554},{"type":"v1","addr":"172.21.15.103:6823","nonce":2808952554}]},"public_addr":"172.21.15.103:6819/2808952554","cluster_addr":"172.21.15.103:6821/2808952554","heartbeat_back_addr":"172.21.15.103:6825/2808952554","heartbeat_front_addr":"172.21.15.103:6823/2808952554","state":["exists","up"]},{"osd":7,"uuid":"c96851b2-9ad0-4adb-b840-9bd11ab6655e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6817","nonce":3442676613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6819","nonce":3442676613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6823","nonce":3442676613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3442676613},{"type":"v1","addr":"172.21.15.139:6821","nonce":3442676613}]},"public_addr":"172.21.15.139:6817/3442676613","cluster_addr":"172.21.15.139:6819/3442676613","heartbeat_back_addr":"172.21.15.139:6823/3442676613","heartbeat_front_addr":"172.21.15.139:6821/3442676613","state":["exists","up"]},{"osd":8,"uuid":"f5b69841-792a-4d17-8406-278c8d11a20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6817","nonce":639772211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6819","nonce":639772211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6823","nonce":639772211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":639772211},{"type":"v1","addr":"172.21.15.167:6821","nonce":639772211}]},"public_addr":"172.21.15.167:6817/639772211","cluster_addr":"172.21.15.167:6819/639772211","heartbeat_back_addr":"172.21.15.167:6823/639772211","heartbeat_front_addr":"172.21.15.167:6821/639772211","state":["exists","up"]},{"osd":9,"uuid":"0e609bbd-4a74-489f-8b8b-47bb13175b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6827","nonce":3249722651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6829","nonce":3249722651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6833","nonce":3249722651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3249722651},{"type":"v1","addr":"172.21.15.103:6831","nonce":3249722651}]},"public_addr":"172.21.15.103:6827/3249722651","cluster_addr":"172.21.15.103:6829/3249722651","heartbeat_back_addr":"172.21.15.103:6833/3249722651","heartbeat_front_addr":"172.21.15.103:6831/3249722651","state":["exists","up"]},{"osd":10,"uuid":"52a68fc4-9a55-4a43-a1da-72019facc4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6825","nonce":3256384109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6827","nonce":3256384109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6831","nonce":3256384109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3256384109},{"type":"v1","addr":"172.21.15.139:6829","nonce":3256384109}]},"public_addr":"172.21.15.139:6825/3256384109","cluster_addr":"172.21.15.139:6827/3256384109","heartbeat_back_addr":"172.21.15.139:6831/3256384109","heartbeat_front_addr":"172.21.15.139:6829/3256384109","state":["exists","up"]},{"osd":11,"uuid":"356aa3c1-baca-473e-a6fc-cf366d4783a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6825","nonce":259212791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6827","nonce":259212791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6831","nonce":259212791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":259212791},{"type":"v1","addr":"172.21.15.167:6829","nonce":259212791}]},"public_addr":"172.21.15.167:6825/259212791","cluster_addr":"172.21.15.167:6827/259212791","heartbeat_back_addr":"172.21.15.167:6831/259212791","heartbeat_front_addr":"172.21.15.167:6829/259212791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:54:45.268935+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:09.396004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:32.811272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:55:58.496586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:22.021042+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:56:47.411017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:11.144830+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:57:36.140551+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:00.693697+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:26.589955+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:58:52.752315+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:59:17.957566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6801/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:6800/2384335573":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/2155492495":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/396485172":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/1467689006":"2024-09-06T01:53:25.001822+0000","172.21.15.103:0/3874104813":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/1573405851":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6801/2132077607":"2024-09-06T01:53:03.000202+0000","172.21.15.103:0/3892241075":"2024-09-06T01:53:03.000202+0000","172.21.15.103:6800/2132077607":"2024-09-06T01:53:03.000202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T02:10:23.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd07c0a2b80 msgr2=0x7fd07c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:23.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd07c0a2b80 0x7fd07c0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd088006fd0 tx=0x7fd088008040 comp rx=0 tx=0).stop 2024-09-05T02:10:23.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 msgr2=0x7fd0940744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:23.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 0x7fd0940744f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd084006010 tx=0x7fd084004270 comp rx=0 tx=0).stop 2024-09-05T02:10:23.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 shutdown_connections 2024-09-05T02:10:23.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd094106f10 0x7fd094074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:23.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fd07c0a2b80 0x7fd07c0a5040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:23.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094102790 0x7fd094074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:23.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 --2- 172.21.15.103:0/1661898354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd094101de0 0x7fd0940744f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:23.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 >> 172.21.15.103:0/1661898354 conn(0x7fd0940fd8c0 msgr2=0x7fd0940fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:23.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 shutdown_connections 2024-09-05T02:10:23.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:23.056+0000 7fd098bf7640 1 -- 172.21.15.103:0/1661898354 wait complete. 2024-09-05T02:10:23.659 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:23 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1661898354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:10:23.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:23 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1661898354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:10:23.660 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:23 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1661898354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T02:10:23.733 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs fail cephfs 2024-09-05T02:10:23.980 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:24.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-mon[30902]: pgmap v589: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-05T02:10:24.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.469+0000 7fa4cf810640 1 -- 172.21.15.103:0/2897288905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102570 msgr2=0x7fa4c81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:24.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.469+0000 7fa4cf810640 1 --2- 172.21.15.103:0/2897288905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102570 0x7fa4c81029b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009900 tx=0x7fa4c402a480 comp rx=0 tx=0).stop 2024-09-05T02:10:24.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 -- 172.21.15.103:0/2897288905 shutdown_connections 2024-09-05T02:10:24.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 --2- 172.21.15.103:0/2897288905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c8102ef0 0x7fa4c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:24.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 --2- 172.21.15.103:0/2897288905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102570 0x7fa4c81029b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:24.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 --2- 172.21.15.103:0/2897288905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4c8100e20 0x7fa4c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:24.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 -- 172.21.15.103:0/2897288905 >> 172.21.15.103:0/2897288905 conn(0x7fa4c80fc7d0 msgr2=0x7fa4c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:24.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 -- 172.21.15.103:0/2897288905 shutdown_connections 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 -- 172.21.15.103:0/2897288905 wait complete. 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 Processor -- start 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 -- start start 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4c8100e20 0x7fa4c8199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c8102570 0x7fa4c8199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102ef0 0x7fa4c81a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8110ed0 con 0x7fa4c8102ef0 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.470+0000 7fa4cf810640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8110d50 con 0x7fa4c8100e20 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cf810640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8111050 con 0x7fa4c8102570 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cddad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102ef0 0x7fa4c81a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cddad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102ef0 0x7fa4c81a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35680/0 (socket says 172.21.15.103:35680) 2024-09-05T02:10:24.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cddad640 1 -- 172.21.15.103:0/951415733 learned_addr learned my addr 172.21.15.103:0/951415733 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cd5ac640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4c8100e20 0x7fa4c8199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4ccdab640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c8102570 0x7fa4c8199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cddad640 1 -- 172.21.15.103:0/951415733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c8102570 msgr2=0x7fa4c8199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cddad640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c8102570 0x7fa4c8199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cddad640 1 -- 172.21.15.103:0/951415733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4c8100e20 msgr2=0x7fa4c8199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cddad640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4c8100e20 0x7fa4c8199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cddad640 1 -- 172.21.15.103:0/951415733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c81a4900 con 0x7fa4c8102ef0 2024-09-05T02:10:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cddad640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102ef0 0x7fa4c81a43c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000b970 tx=0x7fa4c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4ba7fc640 1 -- 172.21.15.103:0/951415733 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0019070 con 0x7fa4c8102ef0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4ba7fc640 1 -- 172.21.15.103:0/951415733 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4c0014440 con 0x7fa4c8102ef0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4ba7fc640 1 -- 172.21.15.103:0/951415733 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c00075d0 con 0x7fa4c8102ef0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c81a4ad0 con 0x7fa4c8102ef0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.471+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c81a4f90 con 0x7fa4c8102ef0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.472+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c8101200 con 0x7fa4c8102ef0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.474+0000 7fa4ba7fc640 1 -- 172.21.15.103:0/951415733 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa4c001c070 con 0x7fa4c8102ef0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.474+0000 7fa4ba7fc640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4b40a2c00 0x7fa4b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.474+0000 7fa4cd5ac640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4b40a2c00 0x7fa4b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.474+0000 7fa4ba7fc640 1 -- 172.21.15.103:0/951415733 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa4c00ced80 con 0x7fa4c8102ef0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.475+0000 7fa4cd5ac640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4b40a2c00 0x7fa4b40a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009990 tx=0x7fa4bc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.477+0000 7fa4ba7fc640 1 -- 172.21.15.103:0/951415733 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4c009b930 con 0x7fa4c8102ef0 2024-09-05T02:10:24.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-mon[35122]: pgmap v589: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-05T02:10:24.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-mon[34262]: pgmap v589: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-05T02:10:24.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.681+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa4c806a560 con 0x7fa4c8102ef0 2024-09-05T02:10:24.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.684+0000 7fa4ba7fc640 1 -- 172.21.15.103:0/951415733 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7fa4c806a560 con 0x7fa4c8102ef0 2024-09-05T02:10:24.684 INFO:teuthology.orchestra.run.smithi103.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-05T02:10:24.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4b40a2c00 msgr2=0x7fa4b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:24.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4b40a2c00 0x7fa4b40a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009990 tx=0x7fa4bc009110 comp rx=0 tx=0).stop 2024-09-05T02:10:24.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102ef0 msgr2=0x7fa4c81a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:24.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102ef0 0x7fa4c81a43c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000b970 tx=0x7fa4c000bd70 comp rx=0 tx=0).stop 2024-09-05T02:10:24.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 shutdown_connections 2024-09-05T02:10:24.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4c8102ef0 0x7fa4c81a43c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:24.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fa4b40a2c00 0x7fa4b40a50c0 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009990 tx=0x7fa4bc009110 comp rx=0 tx=0).stop 2024-09-05T02:10:24.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c8102570 0x7fa4c8199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:24.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 --2- 172.21.15.103:0/951415733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4c8100e20 0x7fa4c8199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:24.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.687+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 >> 172.21.15.103:0/951415733 conn(0x7fa4c80fc7d0 msgr2=0x7fa4c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:24.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.688+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 shutdown_connections 2024-09-05T02:10:24.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:24.688+0000 7fa4cf810640 1 -- 172.21.15.103:0/951415733 wait complete. 2024-09-05T02:10:25.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a[30898]: 2024-09-05T02:10:24.682+0000 7f10fee42640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T02:10:25.132 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a[69086]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.132 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a[69086]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.132 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-mds[69093]: -- 172.21.15.103:0/2871592196 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e03f4e7c20 con 0x55e03f4c5400 2024-09-05T02:10:25.132 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a[69086]: starting mds.a at 2024-09-05T02:10:25.133 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-g[68676]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.133 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-g[68676]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.133 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-mds[68680]: -- 172.21.15.103:0/563037023 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56325dc35860 con 0x56325dc12800 2024-09-05T02:10:25.133 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:10:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-g[68676]: starting mds.g at 2024-09-05T02:10:25.135 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f[65731]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.135 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f[65731]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.136 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f[65731]: starting mds.f at 2024-09-05T02:10:25.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l[65425]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l[65425]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-mds[65429]: -- 172.21.15.167:0/4030088092 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562adfcbd860 con 0x562adfc9b400 2024-09-05T02:10:25.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l[65425]: starting mds.l at 2024-09-05T02:10:25.136 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c[66030]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.137 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c[66030]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.137 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-mds[66034]: -- 172.21.15.167:0/950198175 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a4ffc5860 con 0x560a4ffa3000 2024-09-05T02:10:25.137 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c[66030]: starting mds.c at 2024-09-05T02:10:25.137 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i[66332]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.137 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i[66332]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.137 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-mds[66336]: -- 172.21.15.167:0/2365482891 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596bf759860 con 0x5596bf736800 2024-09-05T02:10:25.137 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:10:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i[66332]: starting mds.i at 2024-09-05T02:10:25.146 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b[65040]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.146 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b[65040]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.146 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-mds[65044]: -- 172.21.15.139:0/3999840639 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646091f5860 con 0x5646091d2800 2024-09-05T02:10:25.146 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b[65040]: starting mds.b at 2024-09-05T02:10:25.146 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e[64436]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.146 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e[64436]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.147 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-mds[64440]: -- 172.21.15.139:0/3685163389 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d1689fc20 con 0x563d1687c800 2024-09-05T02:10:25.147 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e[64436]: starting mds.e at 2024-09-05T02:10:25.147 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h[64739]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.147 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h[64739]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.147 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-mds[64743]: -- 172.21.15.139:0/1234029851 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559a4ceae680 con 0x559a4ce83400 2024-09-05T02:10:25.147 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-mds[64743]: -- 172.21.15.139:0/1234029851 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a4cea5860 con 0x559a4ce83400 2024-09-05T02:10:25.147 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h[64739]: starting mds.h at 2024-09-05T02:10:25.148 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k[64123]: ignoring --setuser ceph since I am not root 2024-09-05T02:10:25.148 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k[64123]: ignoring --setgroup ceph since I am not root 2024-09-05T02:10:25.148 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-mds[64127]: -- 172.21.15.139:0/1990775804 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558be177860 con 0x5558be155000 2024-09-05T02:10:25.148 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:10:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k[64123]: starting mds.k at 2024-09-05T02:10:25.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:25 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/951415733' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T02:10:25.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:25 smithi103 ceph-mon[30902]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T02:10:25.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:25 smithi103 ceph-mon[30902]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T02:10:25.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:25 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/951415733' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T02:10:25.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:25 smithi103 ceph-mon[30902]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T02:10:25.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:25 smithi103 ceph-mon[30902]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-05T02:10:25.393 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-05T02:10:25.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:25 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/951415733' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T02:10:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:25 smithi167 ceph-mon[35122]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T02:10:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:25 smithi167 ceph-mon[35122]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T02:10:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:25 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/951415733' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T02:10:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:25 smithi167 ceph-mon[35122]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T02:10:25.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:25 smithi167 ceph-mon[35122]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-05T02:10:25.639 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:25 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/951415733' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T02:10:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:25 smithi139 ceph-mon[34262]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T02:10:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:25 smithi139 ceph-mon[34262]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T02:10:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:25 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/951415733' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T02:10:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:25 smithi139 ceph-mon[34262]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T02:10:25.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:25 smithi139 ceph-mon[34262]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-05T02:10:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 -- 172.21.15.103:0/56053434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba410c170 msgr2=0x7f1ba410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 --2- 172.21.15.103:0/56053434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba410c170 0x7f1ba410e610 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009ca0 tx=0x7f1b9c02a950 comp rx=0 tx=0).stop 2024-09-05T02:10:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 -- 172.21.15.103:0/56053434 shutdown_connections 2024-09-05T02:10:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 --2- 172.21.15.103:0/56053434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba410c170 0x7f1ba410e610 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 --2- 172.21.15.103:0/56053434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba4103660 0x7f1ba410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 --2- 172.21.15.103:0/56053434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba4102d40 0x7f1ba4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 -- 172.21.15.103:0/56053434 >> 172.21.15.103:0/56053434 conn(0x7f1ba40fc7a0 msgr2=0x7f1ba40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 -- 172.21.15.103:0/56053434 shutdown_connections 2024-09-05T02:10:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 -- 172.21.15.103:0/56053434 wait complete. 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 Processor -- start 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.139+0000 7f1bac534640 1 -- start start 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1bac534640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba4102d40 0x7f1ba4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1bac534640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba4103660 0x7f1ba4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1bac534640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba410c170 0x7f1ba419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1bac534640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4111e20 con 0x7f1ba4102d40 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1bac534640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4111ca0 con 0x7f1ba410c170 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1bac534640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4111fa0 con 0x7f1ba4103660 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baa2d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba4102d40 0x7f1ba4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baa2d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba4102d40 0x7f1ba4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35754/0 (socket says 172.21.15.103:35754) 2024-09-05T02:10:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baa2d0640 1 -- 172.21.15.103:0/2699945581 learned_addr learned my addr 172.21.15.103:0/2699945581 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1ba9acf640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba4103660 0x7f1ba4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baaad1640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba410c170 0x7f1ba419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baa2d0640 1 -- 172.21.15.103:0/2699945581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba4103660 msgr2=0x7f1ba4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baa2d0640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba4103660 0x7f1ba4198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baa2d0640 1 -- 172.21.15.103:0/2699945581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba410c170 msgr2=0x7f1ba419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baa2d0640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba410c170 0x7f1ba419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baa2d0640 1 -- 172.21.15.103:0/2699945581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba419d2b0 con 0x7f1ba4102d40 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1ba9acf640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba4103660 0x7f1ba4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1baa2d0640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba4102d40 0x7f1ba4198300 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c890 tx=0x7f1ba000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1b9b7fe640 1 -- 172.21.15.103:0/2699945581 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0004220 con 0x7f1ba4102d40 2024-09-05T02:10:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1b9b7fe640 1 -- 172.21.15.103:0/2699945581 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba00043c0 con 0x7f1ba4102d40 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1b9b7fe640 1 -- 172.21.15.103:0/2699945581 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0004a00 con 0x7f1ba4102d40 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.140+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba419d5a0 con 0x7f1ba4102d40 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.141+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba41a4e80 con 0x7f1ba4102d40 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.141+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba4103120 con 0x7f1ba4102d40 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.142+0000 7f1b9b7fe640 1 -- 172.21.15.103:0/2699945581 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1ba0004ba0 con 0x7f1ba4102d40 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.143+0000 7f1b9b7fe640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b900a2b80 0x7f1b900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.143+0000 7f1ba9acf640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b900a2b80 0x7f1b900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.143+0000 7f1b9b7fe640 1 -- 172.21.15.103:0/2699945581 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9609+0+0 (secure 0 0 0) 0x7f1ba00cf310 con 0x7f1ba4102d40 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.144+0000 7f1ba9acf640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b900a2b80 0x7f1b900a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005fd0 tx=0x7f1b94005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.147+0000 7f1b9b7fe640 1 -- 172.21.15.103:0/2699945581 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ba0014030 con 0x7f1ba4102d40 2024-09-05T02:10:26.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:26.349+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1ba402d7c0 con 0x7f1ba4102d40 2024-09-05T02:10:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: pgmap v591: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.103:6838/3111492758,v1:172.21.15.103:6839/3111492758] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.103:6840/2992777724,v1:172.21.15.103:6841/2992777724] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6833/1891439850,v1:172.21.15.167:6834/1891439850] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6832/4003477065,v1:172.21.15.139:6834/4003477065] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6832/349530552,v1:172.21.15.167:6835/349530552] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6833/164062785,v1:172.21.15.139:6835/164062785] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6837/4156988880,v1:172.21.15.139:6839/4156988880] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6836/3493104444,v1:172.21.15.167:6838/3493104444] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.167:6837/1480121155,v1:172.21.15.167:6839/1480121155] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: mds.? [v2:172.21.15.139:6836/827032970,v1:172.21.15.139:6838/827032970] up:boot 2024-09-05T02:10:26.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T02:10:26.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:26 smithi167 ceph-mon[35122]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T02:10:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: pgmap v591: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-05T02:10:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.103:6838/3111492758,v1:172.21.15.103:6839/3111492758] up:boot 2024-09-05T02:10:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.103:6840/2992777724,v1:172.21.15.103:6841/2992777724] up:boot 2024-09-05T02:10:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6833/1891439850,v1:172.21.15.167:6834/1891439850] up:boot 2024-09-05T02:10:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6832/4003477065,v1:172.21.15.139:6834/4003477065] up:boot 2024-09-05T02:10:26.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6832/349530552,v1:172.21.15.167:6835/349530552] up:boot 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6833/164062785,v1:172.21.15.139:6835/164062785] up:boot 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6837/4156988880,v1:172.21.15.139:6839/4156988880] up:boot 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6836/3493104444,v1:172.21.15.167:6838/3493104444] up:boot 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.167:6837/1480121155,v1:172.21.15.167:6839/1480121155] up:boot 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: mds.? [v2:172.21.15.139:6836/827032970,v1:172.21.15.139:6838/827032970] up:boot 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T02:10:26.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T02:10:26.648 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T02:10:26.648 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:26 smithi139 ceph-mon[34262]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: pgmap v591: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.103:6838/3111492758,v1:172.21.15.103:6839/3111492758] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.103:6840/2992777724,v1:172.21.15.103:6841/2992777724] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6833/1891439850,v1:172.21.15.167:6834/1891439850] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6832/4003477065,v1:172.21.15.139:6834/4003477065] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6832/349530552,v1:172.21.15.167:6835/349530552] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6833/164062785,v1:172.21.15.139:6835/164062785] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6837/4156988880,v1:172.21.15.139:6839/4156988880] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6836/3493104444,v1:172.21.15.167:6838/3493104444] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.167:6837/1480121155,v1:172.21.15.167:6839/1480121155] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: mds.? [v2:172.21.15.139:6836/827032970,v1:172.21.15.139:6838/827032970] up:boot 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-05T02:10:26.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T02:10:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T02:10:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T02:10:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T02:10:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T02:10:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T02:10:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T02:10:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T02:10:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T02:10:26.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:26 smithi103 ceph-mon[30902]: from='mgr.14150 172.21.15.103:0/1821705650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T02:10:27.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.352+0000 7f1b9b7fe640 1 -- 172.21.15.103:0/2699945581 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f1ba009bd10 con 0x7f1ba4102d40 2024-09-05T02:10:27.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b900a2b80 msgr2=0x7f1b900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:27.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b900a2b80 0x7f1b900a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005fd0 tx=0x7f1b94005850 comp rx=0 tx=0).stop 2024-09-05T02:10:27.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba4102d40 msgr2=0x7f1ba4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:27.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba4102d40 0x7f1ba4198300 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c890 tx=0x7f1ba000cd50 comp rx=0 tx=0).stop 2024-09-05T02:10:27.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 shutdown_connections 2024-09-05T02:10:27.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba410c170 0x7f1ba419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:27.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f1b900a2b80 0x7f1b900a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:27.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba4103660 0x7f1ba4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:27.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 --2- 172.21.15.103:0/2699945581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ba4102d40 0x7f1ba4198300 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:27.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 >> 172.21.15.103:0/2699945581 conn(0x7f1ba40fc7a0 msgr2=0x7f1ba40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:27.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 shutdown_connections 2024-09-05T02:10:27.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:27.354+0000 7f1bac534640 1 -- 172.21.15.103:0/2699945581 wait complete. 2024-09-05T02:10:27.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:27 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2699945581' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T02:10:27.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:27 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2699945581' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T02:10:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:27 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2699945581' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T02:10:28.066 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-05T02:10:28.320 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:28.620 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:28 smithi103 ceph-mon[30902]: pgmap v592: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-05T02:10:28.620 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:28 smithi103 ceph-mon[30902]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T02:10:28.620 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:28 smithi103 ceph-mon[30902]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T02:10:28.620 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:28 smithi103 ceph-mon[30902]: Cluster is now healthy 2024-09-05T02:10:28.620 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:28 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/2699945581' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T02:10:28.620 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:28 smithi103 ceph-mon[30902]: fsmap 12 up:standby 2024-09-05T02:10:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:28 smithi167 ceph-mon[35122]: pgmap v592: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-05T02:10:28.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:28 smithi167 ceph-mon[35122]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T02:10:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:28 smithi167 ceph-mon[35122]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T02:10:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:28 smithi167 ceph-mon[35122]: Cluster is now healthy 2024-09-05T02:10:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:28 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/2699945581' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T02:10:28.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:28 smithi167 ceph-mon[35122]: fsmap 12 up:standby 2024-09-05T02:10:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:28 smithi139 ceph-mon[34262]: pgmap v592: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-05T02:10:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:28 smithi139 ceph-mon[34262]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T02:10:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:28 smithi139 ceph-mon[34262]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T02:10:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:28 smithi139 ceph-mon[34262]: Cluster is now healthy 2024-09-05T02:10:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:28 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/2699945581' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T02:10:28.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:28 smithi139 ceph-mon[34262]: fsmap 12 up:standby 2024-09-05T02:10:28.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.802+0000 7ff449047640 1 -- 172.21.15.103:0/4126407506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444101420 msgr2=0x7ff44410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:28.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.802+0000 7ff449047640 1 --2- 172.21.15.103:0/4126407506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444101420 0x7ff44410d950 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009cd0 tx=0x7ff43c02a8e0 comp rx=0 tx=0).stop 2024-09-05T02:10:28.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.802+0000 7ff449047640 1 -- 172.21.15.103:0/4126407506 shutdown_connections 2024-09-05T02:10:28.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.802+0000 7ff449047640 1 --2- 172.21.15.103:0/4126407506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444101420 0x7ff44410d950 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:28.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.802+0000 7ff449047640 1 --2- 172.21.15.103:0/4126407506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff444100a80 0x7ff444100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.802+0000 7ff449047640 1 --2- 172.21.15.103:0/4126407506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff444106a80 0x7ff444106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.802+0000 7ff449047640 1 -- 172.21.15.103:0/4126407506 >> 172.21.15.103:0/4126407506 conn(0x7ff4440fc7a0 msgr2=0x7ff4440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.802+0000 7ff449047640 1 -- 172.21.15.103:0/4126407506 shutdown_connections 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.802+0000 7ff449047640 1 -- 172.21.15.103:0/4126407506 wait complete. 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.803+0000 7ff449047640 1 Processor -- start 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.803+0000 7ff449047640 1 -- start start 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.803+0000 7ff449047640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444100a80 0x7ff4441983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.803+0000 7ff449047640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff444101420 0x7ff4441988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.803+0000 7ff449047640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff444106a80 0x7ff44419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.803+0000 7ff449047640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff444110d30 con 0x7ff444100a80 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.803+0000 7ff449047640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff444110bb0 con 0x7ff444101420 2024-09-05T02:10:28.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.803+0000 7ff449047640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff444110eb0 con 0x7ff444106a80 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff442d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444100a80 0x7ff4441983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff442d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444100a80 0x7ff4441983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35782/0 (socket says 172.21.15.103:35782) 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff442d9d640 1 -- 172.21.15.103:0/188952635 learned_addr learned my addr 172.21.15.103:0/188952635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff44259c640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff444101420 0x7ff4441988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff44359e640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff444106a80 0x7ff44419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff442d9d640 1 -- 172.21.15.103:0/188952635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff444106a80 msgr2=0x7ff44419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff442d9d640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff444106a80 0x7ff44419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff442d9d640 1 -- 172.21.15.103:0/188952635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff444101420 msgr2=0x7ff4441988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff442d9d640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff444101420 0x7ff4441988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff442d9d640 1 -- 172.21.15.103:0/188952635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44419d350 con 0x7ff444100a80 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff442d9d640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444100a80 0x7ff4441983a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff43400d770 tx=0x7ff43400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:28.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff42bfff640 1 -- 172.21.15.103:0/188952635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434017070 con 0x7ff444100a80 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44419d640 con 0x7ff444100a80 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff42bfff640 1 -- 172.21.15.103:0/188952635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4340054d0 con 0x7ff444100a80 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff42bfff640 1 -- 172.21.15.103:0/188952635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434004cf0 con 0x7ff444100a80 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.804+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4441a4f20 con 0x7ff444100a80 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.805+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff444100ee0 con 0x7ff444100a80 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.806+0000 7ff42bfff640 1 -- 172.21.15.103:0/188952635 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff434004e90 con 0x7ff444100a80 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.807+0000 7ff42bfff640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff42c0a2b80 0x7ff42c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.807+0000 7ff44259c640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff42c0a2b80 0x7ff42c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.807+0000 7ff42bfff640 1 -- 172.21.15.103:0/188952635 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9609+0+0 (secure 0 0 0) 0x7ff4340cf360 con 0x7ff444100a80 2024-09-05T02:10:28.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.807+0000 7ff44259c640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff42c0a2b80 0x7ff42c0a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff4441051d0 tx=0x7ff43800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:28.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.811+0000 7ff42bfff640 1 -- 172.21.15.103:0/188952635 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff43409b6b0 con 0x7ff444100a80 2024-09-05T02:10:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:28.977+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff44410a000 con 0x7ff444100a80 2024-09-05T02:10:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.361+0000 7ff42bfff640 1 -- 172.21.15.103:0/188952635 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7ff43409f9b0 con 0x7ff444100a80 2024-09-05T02:10:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' removed 2024-09-05T02:10:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff42c0a2b80 msgr2=0x7ff42c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff42c0a2b80 0x7ff42c0a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff4441051d0 tx=0x7ff43800b390 comp rx=0 tx=0).stop 2024-09-05T02:10:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444100a80 msgr2=0x7ff4441983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444100a80 0x7ff4441983a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff43400d770 tx=0x7ff43400dc30 comp rx=0 tx=0).stop 2024-09-05T02:10:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 shutdown_connections 2024-09-05T02:10:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7ff42c0a2b80 0x7ff42c0a5040 secure :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff4441051d0 tx=0x7ff43800b390 comp rx=0 tx=0).stop 2024-09-05T02:10:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff444106a80 0x7ff44419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff444101420 0x7ff4441988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 --2- 172.21.15.103:0/188952635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff444100a80 0x7ff4441983a0 secure :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff43400d770 tx=0x7ff43400dc30 comp rx=0 tx=0).stop 2024-09-05T02:10:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.364+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 >> 172.21.15.103:0/188952635 conn(0x7ff4440fc7a0 msgr2=0x7ff44406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.365+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 shutdown_connections 2024-09-05T02:10:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:29.365+0000 7ff449047640 1 -- 172.21.15.103:0/188952635 wait complete. 2024-09-05T02:10:29.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:29 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/188952635' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T02:10:29.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:29 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/188952635' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T02:10:29.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:29 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/188952635' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T02:10:30.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-05T02:10:30.342 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:30.623 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:30 smithi103 ceph-mon[30902]: pgmap v593: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-09-05T02:10:30.624 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:30 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/188952635' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T02:10:30.624 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:30 smithi103 ceph-mon[30902]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T02:10:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:30 smithi167 ceph-mon[35122]: pgmap v593: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-09-05T02:10:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:30 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/188952635' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T02:10:30.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:30 smithi167 ceph-mon[35122]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T02:10:30.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:30 smithi139 ceph-mon[34262]: pgmap v593: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-09-05T02:10:30.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:30 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/188952635' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T02:10:30.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:30 smithi139 ceph-mon[34262]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T02:10:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 -- 172.21.15.103:0/3089417494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 msgr2=0x7fe33010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 --2- 172.21.15.103:0/3089417494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 0x7fe33010d950 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe328009cd0 tx=0x7fe32802a8e0 comp rx=0 tx=0).stop 2024-09-05T02:10:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 -- 172.21.15.103:0/3089417494 shutdown_connections 2024-09-05T02:10:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 --2- 172.21.15.103:0/3089417494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 0x7fe33010d950 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 --2- 172.21.15.103:0/3089417494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe330100a80 0x7fe330100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 --2- 172.21.15.103:0/3089417494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe330106a80 0x7fe330106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 -- 172.21.15.103:0/3089417494 >> 172.21.15.103:0/3089417494 conn(0x7fe3300fc7a0 msgr2=0x7fe3300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 -- 172.21.15.103:0/3089417494 shutdown_connections 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 -- 172.21.15.103:0/3089417494 wait complete. 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.835+0000 7fe335715640 1 Processor -- start 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe335715640 1 -- start start 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe335715640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe330100a80 0x7fe330198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe335715640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 0x7fe3301988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe335715640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe330106a80 0x7fe33019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe335715640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe330110d30 con 0x7fe330101420 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe335715640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe330110bb0 con 0x7fe330100a80 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe335715640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe330110eb0 con 0x7fe330106a80 2024-09-05T02:10:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 0x7fe3301988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 0x7fe3301988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48412/0 (socket says 172.21.15.103:48412) 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32e7fc640 1 -- 172.21.15.103:0/891657113 learned_addr learned my addr 172.21.15.103:0/891657113 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32f7fe640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe330106a80 0x7fe33019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32effd640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe330100a80 0x7fe330198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32e7fc640 1 -- 172.21.15.103:0/891657113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe330106a80 msgr2=0x7fe33019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32e7fc640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe330106a80 0x7fe33019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32e7fc640 1 -- 172.21.15.103:0/891657113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe330100a80 msgr2=0x7fe330198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32e7fc640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe330100a80 0x7fe330198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.836+0000 7fe32e7fc640 1 -- 172.21.15.103:0/891657113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33019d320 con 0x7fe330101420 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.837+0000 7fe32e7fc640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 0x7fe3301988b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe32400c890 tx=0x7fe32400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.837+0000 7fe317fff640 1 -- 172.21.15.103:0/891657113 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324004220 con 0x7fe330101420 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.837+0000 7fe317fff640 1 -- 172.21.15.103:0/891657113 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3240043c0 con 0x7fe330101420 2024-09-05T02:10:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.837+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33019d610 con 0x7fe330101420 2024-09-05T02:10:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.837+0000 7fe317fff640 1 -- 172.21.15.103:0/891657113 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324004a00 con 0x7fe330101420 2024-09-05T02:10:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.837+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3301a4ef0 con 0x7fe330101420 2024-09-05T02:10:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.838+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe330100ee0 con 0x7fe330101420 2024-09-05T02:10:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.840+0000 7fe317fff640 1 -- 172.21.15.103:0/891657113 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe324004ba0 con 0x7fe330101420 2024-09-05T02:10:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.840+0000 7fe317fff640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe3180a2b80 0x7fe3180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.841+0000 7fe32effd640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe3180a2b80 0x7fe3180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.841+0000 7fe317fff640 1 -- 172.21.15.103:0/891657113 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9055+0+0 (secure 0 0 0) 0x7fe3240cf4c0 con 0x7fe330101420 2024-09-05T02:10:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.841+0000 7fe32effd640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe3180a2b80 0x7fe3180a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe320005fd0 tx=0x7fe32000b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:30.844+0000 7fe317fff640 1 -- 172.21.15.103:0/891657113 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe324014030 con 0x7fe330101420 2024-09-05T02:10:31.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.013+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe33002d7c0 con 0x7fe330101420 2024-09-05T02:10:31.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.365+0000 7fe317fff640 1 -- 172.21.15.103:0/891657113 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7fe32409c0f0 con 0x7fe330101420 2024-09-05T02:10:31.367 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' removed 2024-09-05T02:10:31.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.369+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe3180a2b80 msgr2=0x7fe3180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:31.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.369+0000 7fe335715640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe3180a2b80 0x7fe3180a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe320005fd0 tx=0x7fe32000b510 comp rx=0 tx=0).stop 2024-09-05T02:10:31.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.369+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 msgr2=0x7fe3301988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:31.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.369+0000 7fe335715640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 0x7fe3301988b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe32400c890 tx=0x7fe32400cd50 comp rx=0 tx=0).stop 2024-09-05T02:10:31.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.370+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 shutdown_connections 2024-09-05T02:10:31.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.370+0000 7fe335715640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7fe3180a2b80 0x7fe3180a5040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:31.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.370+0000 7fe335715640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe330106a80 0x7fe33019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:31.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.370+0000 7fe335715640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330101420 0x7fe3301988b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:31.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.370+0000 7fe335715640 1 --2- 172.21.15.103:0/891657113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe330100a80 0x7fe330198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:31.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.370+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 >> 172.21.15.103:0/891657113 conn(0x7fe3300fc7a0 msgr2=0x7fe33006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:31.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.376+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 shutdown_connections 2024-09-05T02:10:31.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:31.376+0000 7fe335715640 1 -- 172.21.15.103:0/891657113 wait complete. 2024-09-05T02:10:31.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:31 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/891657113' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T02:10:31.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:31 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/891657113' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T02:10:31.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:31 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/891657113' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T02:10:32.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-05T02:10:32.349 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T02:10:32.613 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:32 smithi103 ceph-mon[30902]: pgmap v595: 97 pgs: 97 active+clean; 577 KiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:10:32.613 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:32 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/891657113' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T02:10:32.613 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:32 smithi103 ceph-mon[30902]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T02:10:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:32 smithi167 ceph-mon[35122]: pgmap v595: 97 pgs: 97 active+clean; 577 KiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:10:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:32 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/891657113' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T02:10:32.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:32 smithi167 ceph-mon[35122]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T02:10:32.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:32 smithi139 ceph-mon[34262]: pgmap v595: 97 pgs: 97 active+clean; 577 KiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:10:32.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:32 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/891657113' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T02:10:32.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:32 smithi139 ceph-mon[34262]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.829+0000 7f653d35f640 1 -- 172.21.15.103:0/2852877409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 msgr2=0x7f65381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.829+0000 7f653d35f640 1 --2- 172.21.15.103:0/2852877409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 0x7f65381067e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f652c009900 tx=0x7f652c02a410 comp rx=0 tx=0).stop 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.829+0000 7f653d35f640 1 -- 172.21.15.103:0/2852877409 shutdown_connections 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.829+0000 7f653d35f640 1 --2- 172.21.15.103:0/2852877409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6538106f10 0x7f653810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.829+0000 7f653d35f640 1 --2- 172.21.15.103:0/2852877409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 0x7f65381067e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.829+0000 7f653d35f640 1 --2- 172.21.15.103:0/2852877409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6538101de0 0x7f65381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.829+0000 7f653d35f640 1 -- 172.21.15.103:0/2852877409 >> 172.21.15.103:0/2852877409 conn(0x7f65380fd8c0 msgr2=0x7f65380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.829+0000 7f653d35f640 1 -- 172.21.15.103:0/2852877409 shutdown_connections 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.829+0000 7f653d35f640 1 -- 172.21.15.103:0/2852877409 wait complete. 2024-09-05T02:10:32.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f653d35f640 1 Processor -- start 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f653d35f640 1 -- start start 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f653d35f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6538101de0 0x7f6538198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f653d35f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 0x7f65381988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f653d35f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6538106f10 0x7f653819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f653d35f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6538111e20 con 0x7f6538102790 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f653d35f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6538111ca0 con 0x7f6538106f10 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f653d35f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6538111fa0 con 0x7f6538101de0 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f65367fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 0x7f65381988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f65367fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 0x7f65381988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48420/0 (socket says 172.21.15.103:48420) 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f65367fc640 1 -- 172.21.15.103:0/1119138200 learned_addr learned my addr 172.21.15.103:0/1119138200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f6536ffd640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6538101de0 0x7f6538198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.830+0000 7f65377fe640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6538106f10 0x7f653819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:32.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f65367fc640 1 -- 172.21.15.103:0/1119138200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6538101de0 msgr2=0x7f6538198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f65367fc640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6538101de0 0x7f6538198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f65367fc640 1 -- 172.21.15.103:0/1119138200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6538106f10 msgr2=0x7f653819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f65367fc640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6538106f10 0x7f653819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f65367fc640 1 -- 172.21.15.103:0/1119138200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653819d330 con 0x7f6538102790 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f65377fe640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6538106f10 0x7f653819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f65367fc640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 0x7f65381988c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f652c005940 tx=0x7f652c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f651ffff640 1 -- 172.21.15.103:0/1119138200 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c03f030 con 0x7f6538102790 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653819d5c0 con 0x7f6538102790 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f651ffff640 1 -- 172.21.15.103:0/1119138200 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f652c003cb0 con 0x7f6538102790 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f651ffff640 1 -- 172.21.15.103:0/1119138200 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c035070 con 0x7f6538102790 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.831+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65381a4f00 con 0x7f6538102790 2024-09-05T02:10:32.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.832+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65381032c0 con 0x7f6538102790 2024-09-05T02:10:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.833+0000 7f651ffff640 1 -- 172.21.15.103:0/1119138200 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f652c042030 con 0x7f6538102790 2024-09-05T02:10:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.833+0000 7f651ffff640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65200a2b80 0x7f65200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T02:10:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.834+0000 7f651ffff640 1 -- 172.21.15.103:0/1119138200 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8496+0+0 (secure 0 0 0) 0x7f652c0ef960 con 0x7f6538102790 2024-09-05T02:10:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.834+0000 7f6536ffd640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65200a2b80 0x7f65200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T02:10:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.834+0000 7f6536ffd640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65200a2b80 0x7f65200a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6528009bd0 tx=0x7f6528005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T02:10:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:32.838+0000 7f651ffff640 1 -- 172.21.15.103:0/1119138200 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f652c0bc7c0 con 0x7f6538102790 2024-09-05T02:10:33.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.005+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f653810a0f0 con 0x7f6538102790 2024-09-05T02:10:33.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.364+0000 7f651ffff640 1 -- 172.21.15.103:0/1119138200 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v94) ==== 178+0+0 (secure 0 0 0) 0x7f652c0bc0c0 con 0x7f6538102790 2024-09-05T02:10:33.366 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data_ec' removed 2024-09-05T02:10:33.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65200a2b80 msgr2=0x7f65200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:33.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65200a2b80 0x7f65200a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6528009bd0 tx=0x7f6528005f90 comp rx=0 tx=0).stop 2024-09-05T02:10:33.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 msgr2=0x7f65381988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T02:10:33.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 0x7f65381988c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f652c005940 tx=0x7f652c0042f0 comp rx=0 tx=0).stop 2024-09-05T02:10:33.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 shutdown_connections 2024-09-05T02:10:33.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6538106f10 0x7f653819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:33.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6538102790 0x7f65381988c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:33.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.103:6800/4267002703,v1:172.21.15.103:6801/4267002703] conn(0x7f65200a2b80 0x7f65200a5040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:33.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 --2- 172.21.15.103:0/1119138200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6538101de0 0x7f6538198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T02:10:33.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.367+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 >> 172.21.15.103:0/1119138200 conn(0x7f65380fd8c0 msgr2=0x7f65380fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T02:10:33.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.368+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 shutdown_connections 2024-09-05T02:10:33.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-05T02:10:33.368+0000 7f653d35f640 1 -- 172.21.15.103:0/1119138200 wait complete. 2024-09-05T02:10:33.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:33 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1119138200' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T02:10:33.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:33 smithi167 ceph-mon[35122]: pgmap v597: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:10:33.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:33 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1119138200' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T02:10:33.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:33 smithi139 ceph-mon[34262]: pgmap v597: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:10:33.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:33 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1119138200' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T02:10:33.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:33 smithi103 ceph-mon[30902]: pgmap v597: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T02:10:34.087 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T02:10:34.114 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T02:10:34.148 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T02:10:34.182 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-05T02:10:34.182 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T02:10:34.198 DEBUG:teuthology.orchestra.run.smithi139:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T02:10:34.216 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T02:10:34.240 INFO:tasks.cephadm:Stopping all daemons... 2024-09-05T02:10:34.241 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-05T02:10:34.241 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.a 2024-09-05T02:10:34.367 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:34 smithi103 systemd[1]: Stopping Ceph mon.a for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:10:34.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:34 smithi139 ceph-mon[34262]: from='client.? 172.21.15.103:0/1119138200' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T02:10:34.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:34 smithi139 ceph-mon[34262]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T02:10:34.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:34 smithi167 ceph-mon[35122]: from='client.? 172.21.15.103:0/1119138200' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T02:10:34.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:34 smithi167 ceph-mon[35122]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T02:10:34.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:34 smithi103 ceph-mon[30902]: from='client.? 172.21.15.103:0/1119138200' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T02:10:34.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:34 smithi103 ceph-mon[30902]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T02:10:34.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:34 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a[30898]: 2024-09-05T02:10:34.554+0000 7f110464d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:10:34.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:34 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a[30898]: 2024-09-05T02:10:34.554+0000 7f110464d640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-05T02:10:35.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:34 smithi103 podman[187478]: 2024-09-05 02:10:34.790373182 +0000 UTC m=+0.357405054 container died 462eb3fe00994516fc228e9775e4f98a2948f15d21be2bc309fe21c85399dddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-05T02:10:35.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:35 smithi103 podman[187478]: 2024-09-05 02:10:35.214052089 +0000 UTC m=+0.781083959 container cleanup 462eb3fe00994516fc228e9775e4f98a2948f15d21be2bc309fe21c85399dddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-05T02:10:35.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:35 smithi103 bash[187478]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a 2024-09-05T02:10:35.735 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 05 02:10:35 smithi103 podman[187490]: 2024-09-05 02:10:35.440721072 +0000 UTC m=+0.648637267 container remove 462eb3fe00994516fc228e9775e4f98a2948f15d21be2bc309fe21c85399dddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2) 2024-09-05T02:10:35.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.a.service' 2024-09-05T02:10:35.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:10:35.839 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-05T02:10:35.840 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-05T02:10:35.840 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.b 2024-09-05T02:10:36.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:35 smithi139 systemd[1]: Stopping Ceph mon.b for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:10:36.646 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-b[34258]: 2024-09-05T02:10:36.168+0000 7f5342382640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:10:36.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-b[34258]: 2024-09-05T02:10:36.168+0000 7f5342382640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-05T02:10:36.647 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:36 smithi139 podman[68747]: 2024-09-05 02:10:36.393721747 +0000 UTC m=+0.358594418 container died 56ef77362ff4162b473bc610f6b2640a73469333552856444a94e45237d0cd2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-05T02:10:37.055 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:36 smithi139 podman[68747]: 2024-09-05 02:10:36.811627859 +0000 UTC m=+0.776500530 container cleanup 56ef77362ff4162b473bc610f6b2640a73469333552856444a94e45237d0cd2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-05T02:10:37.055 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:36 smithi139 bash[68747]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-b 2024-09-05T02:10:37.304 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.b.service' 2024-09-05T02:10:37.326 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:37 smithi139 podman[68758]: 2024-09-05 02:10:37.054795124 +0000 UTC m=+0.658291813 container remove 56ef77362ff4162b473bc610f6b2640a73469333552856444a94e45237d0cd2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-b, ceph=True, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-05T02:10:37.326 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:37 smithi139 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-05T02:10:37.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:37 smithi139 systemd[1]: Stopped Ceph mon.b for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T02:10:37.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 05 02:10:37 smithi139 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.b.service: Consumed 10.328s CPU time. 2024-09-05T02:10:37.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:10:37.360 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-05T02:10:37.360 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-05T02:10:37.360 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.c 2024-09-05T02:10:37.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:37 smithi167 systemd[1]: Stopping Ceph mon.c for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:10:38.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:37 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-c[35118]: 2024-09-05T02:10:37.662+0000 7f42d8ed3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:10:38.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:37 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-c[35118]: 2024-09-05T02:10:37.662+0000 7f42d8ed3640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-05T02:10:38.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:37 smithi167 podman[70474]: 2024-09-05 02:10:37.745217107 +0000 UTC m=+0.200445686 container died f4b9c06b2ea33287dfca271879c89221bdc988cb98701b46b024ce2c9f2ff660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-05T02:10:38.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:38 smithi167 podman[70474]: 2024-09-05 02:10:38.189018687 +0000 UTC m=+0.644247270 container cleanup f4b9c06b2ea33287dfca271879c89221bdc988cb98701b46b024ce2c9f2ff660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-c, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-05T02:10:38.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:38 smithi167 bash[70474]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-c 2024-09-05T02:10:38.658 INFO:journalctl@ceph.mon.c.smithi167.stdout:Sep 05 02:10:38 smithi167 podman[70487]: 2024-09-05 02:10:38.406400601 +0000 UTC m=+0.655689825 container remove f4b9c06b2ea33287dfca271879c89221bdc988cb98701b46b024ce2c9f2ff660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-05T02:10:38.663 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mon.c.service' 2024-09-05T02:10:38.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:10:38.736 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-05T02:10:38.736 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-05T02:10:38.736 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T02:10:39.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:10:38 smithi103 systemd[1]: Stopping Ceph mgr.x for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:10:39.399 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:10:39 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x[31127]: 2024-09-05T02:10:39.052+0000 7f825581f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:10:39.399 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:10:39 smithi103 podman[187613]: 2024-09-05 02:10:39.084500726 +0000 UTC m=+0.158536144 container died a70b6e132a430e02244e7888ee331263d78f62ad55da4e2db17b4940e46520fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-05T02:10:39.898 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:10:39 smithi103 podman[187613]: 2024-09-05 02:10:39.562290007 +0000 UTC m=+0.636325422 container cleanup a70b6e132a430e02244e7888ee331263d78f62ad55da4e2db17b4940e46520fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:10:39.899 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 05 02:10:39 smithi103 bash[187613]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-x 2024-09-05T02:10:40.136 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.x.service' 2024-09-05T02:10:40.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:10:40.172 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-05T02:10:40.172 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-05T02:10:40.172 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.y 2024-09-05T02:10:40.521 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 02:10:40 smithi139 systemd[1]: Stopping Ceph mgr.y for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:10:40.810 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 02:10:40 smithi139 podman[68869]: 2024-09-05 02:10:40.520315238 +0000 UTC m=+0.152694310 container died a84e3496d58952c07f482ffd6bdff8fbf7ed2a79db78e771e642780ba8ea1380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y, ceph=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-05T02:10:41.146 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 02:10:40 smithi139 podman[68869]: 2024-09-05 02:10:40.900295167 +0000 UTC m=+0.532674239 container cleanup a84e3496d58952c07f482ffd6bdff8fbf7ed2a79db78e771e642780ba8ea1380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-91c8a7e, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-05T02:10:41.146 INFO:journalctl@ceph.mgr.y.smithi139.stdout:Sep 05 02:10:40 smithi139 bash[68869]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-y 2024-09-05T02:10:41.338 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.y.service' 2024-09-05T02:10:41.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:10:41.372 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-05T02:10:41.372 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-05T02:10:41.372 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.z 2024-09-05T02:10:41.716 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 02:10:41 smithi167 systemd[1]: Stopping Ceph mgr.z for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:10:42.021 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 02:10:41 smithi167 podman[70598]: 2024-09-05 02:10:41.715254628 +0000 UTC m=+0.144867779 container died 88fae74ca532704254d2f39b6406c796d7ba6544fe141c3ff4a4d5f6935a562d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-05T02:10:42.380 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 02:10:42 smithi167 podman[70598]: 2024-09-05 02:10:42.119966185 +0000 UTC m=+0.549579333 container cleanup 88fae74ca532704254d2f39b6406c796d7ba6544fe141c3ff4a4d5f6935a562d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-05T02:10:42.381 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Sep 05 02:10:42 smithi167 bash[70598]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mgr-z 2024-09-05T02:10:42.546 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mgr.z.service' 2024-09-05T02:10:42.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:10:42.582 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-05T02:10:42.582 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-05T02:10:42.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.0 2024-09-05T02:10:42.899 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:42 smithi103 systemd[1]: Stopping Ceph osd.0 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:10:43.398 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:42 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0[41959]: 2024-09-05T02:10:42.900+0000 7fb4bac0f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:10:43.399 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:42 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0[41959]: 2024-09-05T02:10:42.900+0000 7fb4bac0f640 -1 osd.0 94 *** Got signal Terminated *** 2024-09-05T02:10:43.399 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:42 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0[41959]: 2024-09-05T02:10:42.900+0000 7fb4bac0f640 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:10:48.569 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:48 smithi103 podman[187733]: 2024-09-05 02:10:48.268625255 +0000 UTC m=+5.493159885 container died d9662f06b9d288e0ffb3767ff20497a99e926514526a1154dc290200efa1c7ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-05T02:10:48.899 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:48 smithi103 podman[187733]: 2024-09-05 02:10:48.668391211 +0000 UTC m=+5.892925840 container cleanup d9662f06b9d288e0ffb3767ff20497a99e926514526a1154dc290200efa1c7ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-05T02:10:48.899 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:48 smithi103 bash[187733]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0 2024-09-05T02:10:49.398 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:49 smithi103 podman[187751]: 2024-09-05 02:10:49.025843249 +0000 UTC m=+0.753751248 container remove d9662f06b9d288e0ffb3767ff20497a99e926514526a1154dc290200efa1c7ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-05T02:10:49.725 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:49 smithi103 podman[187840]: 2024-09-05 02:10:49.346412523 +0000 UTC m=+0.015693254 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:10:49.725 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:49 smithi103 podman[187840]: 2024-09-05 02:10:49.479818826 +0000 UTC m=+0.149099590 container create 31baec63e153331d2beb24c48725415147e170efe30cb32258d07660448a039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e) 2024-09-05T02:10:50.149 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:49 smithi103 podman[187840]: 2024-09-05 02:10:49.724827815 +0000 UTC m=+0.394108534 container init 31baec63e153331d2beb24c48725415147e170efe30cb32258d07660448a039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-05T02:10:50.149 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:49 smithi103 podman[187840]: 2024-09-05 02:10:49.728222533 +0000 UTC m=+0.397503252 container start 31baec63e153331d2beb24c48725415147e170efe30cb32258d07660448a039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-05T02:10:50.149 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:49 smithi103 podman[187840]: 2024-09-05 02:10:49.799794382 +0000 UTC m=+0.469075121 container attach 31baec63e153331d2beb24c48725415147e170efe30cb32258d07660448a039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T02:10:50.149 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 05 02:10:49 smithi103 podman[187840]: 2024-09-05 02:10:49.912022338 +0000 UTC m=+0.581303057 container died 31baec63e153331d2beb24c48725415147e170efe30cb32258d07660448a039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD) 2024-09-05T02:10:50.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.0.service' 2024-09-05T02:10:50.610 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:10:50.610 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-05T02:10:50.610 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-05T02:10:50.610 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.1 2024-09-05T02:10:50.896 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:50 smithi139 systemd[1]: Stopping Ceph osd.1 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:10:51.396 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:50 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1[38685]: 2024-09-05T02:10:50.945+0000 7fe474a11640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:10:51.396 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:50 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1[38685]: 2024-09-05T02:10:50.945+0000 7fe474a11640 -1 osd.1 94 *** Got signal Terminated *** 2024-09-05T02:10:51.396 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:50 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1[38685]: 2024-09-05T02:10:50.945+0000 7fe474a11640 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:10:56.598 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:56 smithi139 podman[68995]: 2024-09-05 02:10:56.285247932 +0000 UTC m=+5.479225422 container died 6249536e978db5377066a61cabe03f2ee91e3d542d83e4d8ed17132d861c868a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-05T02:10:56.896 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:56 smithi139 podman[68995]: 2024-09-05 02:10:56.696110489 +0000 UTC m=+5.890087987 container cleanup 6249536e978db5377066a61cabe03f2ee91e3d542d83e4d8ed17132d861c868a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-05T02:10:56.896 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:56 smithi139 bash[68995]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1 2024-09-05T02:10:57.306 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:56 smithi139 podman[69011]: 2024-09-05 02:10:56.956068623 +0000 UTC m=+0.664248370 container remove 6249536e978db5377066a61cabe03f2ee91e3d542d83e4d8ed17132d861c868a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2) 2024-09-05T02:10:57.618 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:57 smithi139 podman[69101]: 2024-09-05 02:10:57.207923779 +0000 UTC m=+0.015893428 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:10:57.619 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:57 smithi139 podman[69101]: 2024-09-05 02:10:57.348957887 +0000 UTC m=+0.156927540 container create c96651bc081960a94e665b087404b280b5aea9f332d82fb69b9597a682caf2aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-05T02:10:57.619 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:57 smithi139 podman[69101]: 2024-09-05 02:10:57.618309037 +0000 UTC m=+0.426278695 container init c96651bc081960a94e665b087404b280b5aea9f332d82fb69b9597a682caf2aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-05T02:10:57.897 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:57 smithi139 podman[69101]: 2024-09-05 02:10:57.621728689 +0000 UTC m=+0.429698338 container start c96651bc081960a94e665b087404b280b5aea9f332d82fb69b9597a682caf2aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-05T02:10:57.897 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:57 smithi139 podman[69101]: 2024-09-05 02:10:57.718466396 +0000 UTC m=+0.526436054 container attach c96651bc081960a94e665b087404b280b5aea9f332d82fb69b9597a682caf2aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:10:57.897 INFO:journalctl@ceph.osd.1.smithi139.stdout:Sep 05 02:10:57 smithi139 podman[69101]: 2024-09-05 02:10:57.8115736 +0000 UTC m=+0.619543249 container died c96651bc081960a94e665b087404b280b5aea9f332d82fb69b9597a682caf2aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-05T02:10:58.522 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.1.service' 2024-09-05T02:10:58.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:10:58.562 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-05T02:10:58.562 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-05T02:10:58.562 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.2 2024-09-05T02:10:58.885 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:10:58 smithi167 systemd[1]: Stopping Ceph osd.2 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:10:59.385 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:10:58 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2[40055]: 2024-09-05T02:10:58.899+0000 7fc3e1a7c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:10:59.386 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:10:58 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2[40055]: 2024-09-05T02:10:58.899+0000 7fc3e1a7c640 -1 osd.2 94 *** Got signal Terminated *** 2024-09-05T02:10:59.386 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:10:58 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2[40055]: 2024-09-05T02:10:58.899+0000 7fc3e1a7c640 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:11:04.634 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:04 smithi167 podman[70722]: 2024-09-05 02:11:04.362960184 +0000 UTC m=+5.604983875 container died 9611305f997b01f0e92001b0f3818d8927ca2a11906f00514443e974cc246d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e) 2024-09-05T02:11:04.885 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:04 smithi167 podman[70722]: 2024-09-05 02:11:04.741397788 +0000 UTC m=+5.983421488 container cleanup 9611305f997b01f0e92001b0f3818d8927ca2a11906f00514443e974cc246d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T02:11:04.885 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:04 smithi167 bash[70722]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2 2024-09-05T02:11:05.354 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:05 smithi167 podman[70740]: 2024-09-05 02:11:05.018009964 +0000 UTC m=+0.654999069 container remove 9611305f997b01f0e92001b0f3818d8927ca2a11906f00514443e974cc246d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-05T02:11:05.636 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:05 smithi167 podman[70829]: 2024-09-05 02:11:05.255823485 +0000 UTC m=+0.015645016 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:11:05.636 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:05 smithi167 podman[70829]: 2024-09-05 02:11:05.377290709 +0000 UTC m=+0.137112275 container create eca73c52668a643f34f0fdbb1052198706a761b45034a6431ac32eaef027d62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T02:11:06.136 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:05 smithi167 podman[70829]: 2024-09-05 02:11:05.638063568 +0000 UTC m=+0.397885090 container init eca73c52668a643f34f0fdbb1052198706a761b45034a6431ac32eaef027d62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2-deactivate, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T02:11:06.136 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:05 smithi167 podman[70829]: 2024-09-05 02:11:05.641549038 +0000 UTC m=+0.401370560 container start eca73c52668a643f34f0fdbb1052198706a761b45034a6431ac32eaef027d62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T02:11:06.136 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:05 smithi167 podman[70829]: 2024-09-05 02:11:05.721597383 +0000 UTC m=+0.481418923 container attach eca73c52668a643f34f0fdbb1052198706a761b45034a6431ac32eaef027d62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-05T02:11:06.136 INFO:journalctl@ceph.osd.2.smithi167.stdout:Sep 05 02:11:05 smithi167 podman[70829]: 2024-09-05 02:11:05.832300855 +0000 UTC m=+0.592122378 container died eca73c52668a643f34f0fdbb1052198706a761b45034a6431ac32eaef027d62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T02:11:06.498 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.2.service' 2024-09-05T02:11:06.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:11:06.540 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-05T02:11:06.540 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-05T02:11:06.540 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.3 2024-09-05T02:11:06.867 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:06 smithi103 systemd[1]: Stopping Ceph osd.3 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:11:07.148 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:06 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3[48470]: 2024-09-05T02:11:06.866+0000 7f2ae8b16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:11:07.149 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:06 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3[48470]: 2024-09-05T02:11:06.866+0000 7f2ae8b16640 -1 osd.3 94 *** Got signal Terminated *** 2024-09-05T02:11:07.149 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:06 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3[48470]: 2024-09-05T02:11:06.866+0000 7f2ae8b16640 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:11:09.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:09 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:09.031+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:10.146 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:10 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:10.052+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:10.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:10 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:10.324+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:10.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:10.564+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:11.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:11 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:11.015+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:11.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:11 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:11.308+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:11.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:11 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:11.614+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:12.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:12 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:12.065+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:12.649 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:12 smithi103 podman[187954]: 2024-09-05 02:11:12.347320298 +0000 UTC m=+5.611779070 container died 507c8e11250a5228172bb992601b48435436db06591d32b835da4691dca3669f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-05T02:11:12.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:12 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:12.356+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:12.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:12 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:12.597+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:13.118 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:12 smithi103 podman[187954]: 2024-09-05 02:11:12.849765489 +0000 UTC m=+6.114224307 container cleanup 507c8e11250a5228172bb992601b48435436db06591d32b835da4691dca3669f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-05T02:11:13.118 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:12 smithi103 bash[187954]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3 2024-09-05T02:11:13.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:13 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:13.044+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:13.398 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:13 smithi103 podman[187970]: 2024-09-05 02:11:13.118125129 +0000 UTC m=+0.769511206 container remove 507c8e11250a5228172bb992601b48435436db06591d32b835da4691dca3669f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T02:11:13.399 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:13 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:13.312+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:13.797 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:13 smithi103 podman[188058]: 2024-09-05 02:11:13.370271161 +0000 UTC m=+0.016165787 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:11:13.798 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:13 smithi103 podman[188058]: 2024-09-05 02:11:13.477461726 +0000 UTC m=+0.123356342 container create a48bc8a0757248bd0afd0f5f82f0d65fce3301b5b5eca550d7d7c52da7789ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-05T02:11:13.798 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:13 smithi103 podman[188058]: 2024-09-05 02:11:13.705331315 +0000 UTC m=+0.351225931 container init a48bc8a0757248bd0afd0f5f82f0d65fce3301b5b5eca550d7d7c52da7789ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-05T02:11:13.798 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:13 smithi103 podman[188058]: 2024-09-05 02:11:13.708649045 +0000 UTC m=+0.354543664 container start a48bc8a0757248bd0afd0f5f82f0d65fce3301b5b5eca550d7d7c52da7789ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-05T02:11:13.798 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:13 smithi103 podman[188058]: 2024-09-05 02:11:13.797107181 +0000 UTC m=+0.443001810 container attach a48bc8a0757248bd0afd0f5f82f0d65fce3301b5b5eca550d7d7c52da7789ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-05T02:11:13.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:13 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:13.575+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:14.149 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 05 02:11:13 smithi103 podman[188058]: 2024-09-05 02:11:13.903329862 +0000 UTC m=+0.549224494 container died a48bc8a0757248bd0afd0f5f82f0d65fce3301b5b5eca550d7d7c52da7789ce0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-05T02:11:14.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:14 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:14.038+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:14.585 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:14 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:14.278+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:14.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.3.service' 2024-09-05T02:11:14.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:11:14.657 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-05T02:11:14.658 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-05T02:11:14.658 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.4 2024-09-05T02:11:14.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:14 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:14.625+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:14.975 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:14 smithi139 systemd[1]: Stopping Ceph osd.4 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:11:15.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:15 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:15.032+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:15.396 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:14 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4[45552]: 2024-09-05T02:11:14.974+0000 7fe6a59d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:11:15.397 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:14 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4[45552]: 2024-09-05T02:11:14.974+0000 7fe6a59d1640 -1 osd.4 94 *** Got signal Terminated *** 2024-09-05T02:11:15.397 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:14 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4[45552]: 2024-09-05T02:11:14.974+0000 7fe6a59d1640 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:11:15.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:15 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:15.258+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:15.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:15 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:15.616+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:16.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:16 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:16.002+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:16.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:16 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:16.281+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:16.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:16 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:16.578+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:16.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:16 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:16.681+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:17.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:16 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:16.863+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:17.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:16 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:16.986+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:17.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:17 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:17.264+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:17.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:17 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:17.264+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:17.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:17 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:17.628+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:17.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:17 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:17.638+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:17.896 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:17 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4[45552]: 2024-09-05T02:11:17.482+0000 7fe69dfdf640 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.151165+0000 front 2024-09-05T02:10:55.151084+0000 (oldest deadline 2024-09-05T02:11:16.851115+0000) 2024-09-05T02:11:18.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:17 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:17.836+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:18.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:18 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:18.022+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:18.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:18 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:18.022+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:18.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:18 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:18.287+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:18.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:18 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:18.287+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:18.896 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:18 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4[45552]: 2024-09-05T02:11:18.461+0000 7fe69dfdf640 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.151165+0000 front 2024-09-05T02:10:55.151084+0000 (oldest deadline 2024-09-05T02:11:16.851115+0000) 2024-09-05T02:11:18.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:18 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:18.535+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:19.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:18 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:18.672+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:19.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:18 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:18.642+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:19.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:18 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:18.790+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:19.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:19 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:19.004+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:19.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:19 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:19.004+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:19.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:19 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:19.295+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:19.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:19 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:19.295+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:19.896 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:19 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4[45552]: 2024-09-05T02:11:19.509+0000 7fe69dfdf640 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.151165+0000 front 2024-09-05T02:10:55.151084+0000 (oldest deadline 2024-09-05T02:11:16.851115+0000) 2024-09-05T02:11:19.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:19 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:19.526+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:20.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:19 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:19.716+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:20.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:19 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:19.659+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:20.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:19 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:19.822+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:20.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:20 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:19.999+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:20.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:20 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:19.999+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:20.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:20.300+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:20.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:20.300+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:20.732 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:20 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:20.629+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:20.733 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:20 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:20.394+0000 7feb2db00640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.898340+0000 front 2024-09-05T02:10:55.898319+0000 (oldest deadline 2024-09-05T02:11:19.998329+0000) 2024-09-05T02:11:20.801 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:20 smithi139 podman[69216]: 2024-09-05 02:11:20.51868338 +0000 UTC m=+5.659460349 container died 129e6d6e46df20993f7066b36f85f9f1f7a15efbf76df91a2de72076d3f4cf6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-05T02:11:20.801 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:20 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:20.554+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:21.137 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:20 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:20.732+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:21.146 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:20 smithi139 podman[69216]: 2024-09-05 02:11:20.899331243 +0000 UTC m=+6.040108204 container cleanup 129e6d6e46df20993f7066b36f85f9f1f7a15efbf76df91a2de72076d3f4cf6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:11:21.146 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:20 smithi139 bash[69216]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4 2024-09-05T02:11:21.146 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:20 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:20.975+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:21.146 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:20 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:20.975+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:21.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:20 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:20.822+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:21.494 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:21 smithi139 podman[69232]: 2024-09-05 02:11:21.192532194 +0000 UTC m=+0.666673882 container remove 129e6d6e46df20993f7066b36f85f9f1f7a15efbf76df91a2de72076d3f4cf6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, ceph=True) 2024-09-05T02:11:21.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:21 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:21.290+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:21.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:21 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:21.290+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:21.711 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:21 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:21.440+0000 7feb2db00640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.898340+0000 front 2024-09-05T02:10:55.898319+0000 (oldest deadline 2024-09-05T02:11:19.998329+0000) 2024-09-05T02:11:21.711 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:21 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:21.591+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:21.771 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:21 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:21.518+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:21.772 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:21 smithi139 podman[69322]: 2024-09-05 02:11:21.493409339 +0000 UTC m=+0.100867548 container create 9486979c892fe65dae7d96e735af7d41e8ecbca014134c521067f38db3f24fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e) 2024-09-05T02:11:21.772 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:21 smithi139 podman[69322]: 2024-09-05 02:11:21.408252178 +0000 UTC m=+0.015710345 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:11:21.772 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:21 smithi139 podman[69322]: 2024-09-05 02:11:21.771243186 +0000 UTC m=+0.378701353 container init 9486979c892fe65dae7d96e735af7d41e8ecbca014134c521067f38db3f24fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-05T02:11:22.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:21 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:21.710+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:22.146 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:22.007+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:22.146 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:22.007+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:22.146 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:21 smithi139 podman[69322]: 2024-09-05 02:11:21.774629227 +0000 UTC m=+0.382087398 container start 9486979c892fe65dae7d96e735af7d41e8ecbca014134c521067f38db3f24fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-05T02:11:22.147 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:21 smithi139 podman[69322]: 2024-09-05 02:11:21.871203685 +0000 UTC m=+0.478661853 container attach 9486979c892fe65dae7d96e735af7d41e8ecbca014134c521067f38db3f24fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-05T02:11:22.147 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 05 02:11:21 smithi139 podman[69322]: 2024-09-05 02:11:21.964246028 +0000 UTC m=+0.571704196 container died 9486979c892fe65dae7d96e735af7d41e8ecbca014134c521067f38db3f24fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-05T02:11:22.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:21 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:21.780+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:22.600 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:22.513+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:22.626 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.4.service' 2024-09-05T02:11:22.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:22 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:22.257+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:22.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:22 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:22.257+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:22.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:11:22.667 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-05T02:11:22.667 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-05T02:11:22.667 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.5 2024-09-05T02:11:22.682 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:22.415+0000 7feb2db00640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.898340+0000 front 2024-09-05T02:10:55.898319+0000 (oldest deadline 2024-09-05T02:11:19.998329+0000) 2024-09-05T02:11:22.682 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:22.561+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:22.991 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:22 smithi167 systemd[1]: Stopping Ceph osd.5 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:11:22.991 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:22.681+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:23.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:22 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:22.820+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:23.385 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:22.989+0000 7feb354f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:11:23.385 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:22.989+0000 7feb354f2640 -1 osd.5 94 *** Got signal Terminated *** 2024-09-05T02:11:23.385 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:22 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:22.989+0000 7feb354f2640 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:11:23.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:22.977+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:23.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:22 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:22.977+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:23.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:23.294+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:23.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:23.294+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:23.730 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:23.393+0000 7feb2db00640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.898340+0000 front 2024-09-05T02:10:55.898319+0000 (oldest deadline 2024-09-05T02:11:19.998329+0000) 2024-09-05T02:11:23.730 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:23.549+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:23.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:23 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:23.515+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:24.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:23 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:23.729+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:24.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:23 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:23.852+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:24.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:23 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:23.991+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:24.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:23 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:23.991+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:24.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:24.329+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:24.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:24.329+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:24.708 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:24.406+0000 7feb2db00640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.898340+0000 front 2024-09-05T02:10:55.898319+0000 (oldest deadline 2024-09-05T02:11:19.998329+0000) 2024-09-05T02:11:24.709 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:24.584+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:24.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:24 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:24.485+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:25.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:24 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:24.707+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:25.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:24.805+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:25.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:25.033+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:25.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:25.033+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:25.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:25.033+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:25.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:25 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:25.367+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:25.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:25 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:25.367+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:25.697 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:25 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:25.407+0000 7feb2db00640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.898340+0000 front 2024-09-05T02:10:55.898319+0000 (oldest deadline 2024-09-05T02:11:19.998329+0000) 2024-09-05T02:11:25.698 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:25 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:25.573+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:25.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:25.493+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:25.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:25 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:25.493+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:26.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:25 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:25.697+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:26.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:25 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:25.784+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:26.398 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:26 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:26.040+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:26.398 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:26 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:26.040+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:26.398 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:26 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:26.040+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:26.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:26.336+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:26.650 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:26.336+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:26.704 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:26 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:26.395+0000 7feb2db00640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.898340+0000 front 2024-09-05T02:10:55.898319+0000 (oldest deadline 2024-09-05T02:11:19.998329+0000) 2024-09-05T02:11:26.704 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:26 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:26.567+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:26.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:26 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:26.479+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:26.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:26 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:26.479+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:27.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:26 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:26.702+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:27.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:26 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:26.750+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:27.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:27 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:27.059+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:27.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:27 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:27.059+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:27.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:27 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:27.059+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:27.635 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:27 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:27.613+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:27.636 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:27 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5[46935]: 2024-09-05T02:11:27.375+0000 7feb2db00640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:55.898340+0000 front 2024-09-05T02:10:55.898319+0000 (oldest deadline 2024-09-05T02:11:19.998329+0000) 2024-09-05T02:11:27.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:27 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:27.371+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:27.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:27 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:27.371+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:27.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:27 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:27.476+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:27.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:27 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:27.476+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:28.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:27 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:27.680+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:28.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:27 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:27.707+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:28.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:27 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:27.707+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.356518+0000 front 2024-09-05T02:11:02.356330+0000 (oldest deadline 2024-09-05T02:11:27.656360+0000) 2024-09-05T02:11:28.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:28 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:28.091+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:28.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:28 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:28.091+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:28.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:28 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:28.091+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:28.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:28 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:28.376+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:28.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:28 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:28.376+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:28.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:28 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:28.376+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:28.723 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:28 smithi167 podman[70946]: 2024-09-05 02:11:28.452590026 +0000 UTC m=+5.593481709 container died 0b46b5de9083c2a458e1397c13a54a6dd281226e06fc5a0387456064429e8dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-05T02:11:28.723 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:28 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:28.599+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:28.723 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:28 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:28.599+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:28.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:28 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:28.467+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:28.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:28 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:28.467+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:29.043 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:28 smithi167 podman[70946]: 2024-09-05 02:11:28.848825517 +0000 UTC m=+5.989717208 container cleanup 0b46b5de9083c2a458e1397c13a54a6dd281226e06fc5a0387456064429e8dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:11:29.043 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:28 smithi167 bash[70946]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5 2024-09-05T02:11:29.043 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:28 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:28.722+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:29.043 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:28 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:28.722+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:29.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:28 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:28.721+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:29.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:28 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:28.721+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.356518+0000 front 2024-09-05T02:11:02.356330+0000 (oldest deadline 2024-09-05T02:11:27.656360+0000) 2024-09-05T02:11:29.385 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:29 smithi167 podman[70962]: 2024-09-05 02:11:29.042039073 +0000 UTC m=+0.592195109 container remove 0b46b5de9083c2a458e1397c13a54a6dd281226e06fc5a0387456064429e8dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e) 2024-09-05T02:11:29.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:29 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:29.083+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:29.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:29 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:29.083+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:29.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:29 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:29.083+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:29.746 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:29 smithi167 podman[71048]: 2024-09-05 02:11:29.369669799 +0000 UTC m=+0.015708782 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:11:29.746 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:29 smithi167 podman[71048]: 2024-09-05 02:11:29.49306403 +0000 UTC m=+0.139103004 container create 90c9d4d3a45be470823bfda8eda522d21131c09af6ecd06a5b24d7ce67d065ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-05T02:11:29.747 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:29 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:29.615+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:29.747 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:29 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:29.615+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:29.766 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:29 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:29.414+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:29.766 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:29 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:29.414+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:29.766 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:29 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:29.414+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:29.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:29 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:29.431+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:29.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:29 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:29.431+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:30.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:29 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:29.753+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:30.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:29 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:29.753+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:30.136 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:29 smithi167 podman[71048]: 2024-09-05 02:11:29.745715466 +0000 UTC m=+0.391754441 container init 90c9d4d3a45be470823bfda8eda522d21131c09af6ecd06a5b24d7ce67d065ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:11:30.136 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:29 smithi167 podman[71048]: 2024-09-05 02:11:29.749118817 +0000 UTC m=+0.395157792 container start 90c9d4d3a45be470823bfda8eda522d21131c09af6ecd06a5b24d7ce67d065ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T02:11:30.136 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:29 smithi167 podman[71048]: 2024-09-05 02:11:29.879316021 +0000 UTC m=+0.525354996 container attach 90c9d4d3a45be470823bfda8eda522d21131c09af6ecd06a5b24d7ce67d065ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-05T02:11:30.136 INFO:journalctl@ceph.osd.5.smithi167.stdout:Sep 05 02:11:29 smithi167 podman[71048]: 2024-09-05 02:11:29.939043193 +0000 UTC m=+0.585082169 container died 90c9d4d3a45be470823bfda8eda522d21131c09af6ecd06a5b24d7ce67d065ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-05T02:11:30.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:29 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:29.764+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:30.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:29 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:29.764+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.356518+0000 front 2024-09-05T02:11:02.356330+0000 (oldest deadline 2024-09-05T02:11:27.656360+0000) 2024-09-05T02:11:30.393 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:30 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:30.114+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:30.393 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:30 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:30.114+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:30.393 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:30 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:30.114+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:30.631 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.5.service' 2024-09-05T02:11:30.646 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:30 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:30.392+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:30.646 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:30 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:30.392+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:30.648 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:30 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:30.383+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:30.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:30 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:30.383+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:30.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:30 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:30.383+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:30.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:11:30.715 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-05T02:11:30.715 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-05T02:11:30.715 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.6 2024-09-05T02:11:30.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:30 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:30.590+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:30.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:30 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:30.590+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:30.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:30 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:30.719+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:30.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:30 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:30.719+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:31.030 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:30 smithi103 systemd[1]: Stopping Ceph osd.6 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:11:31.031 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:30 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:30.776+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:31.031 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:30 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:30.776+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.356518+0000 front 2024-09-05T02:11:02.356330+0000 (oldest deadline 2024-09-05T02:11:27.656360+0000) 2024-09-05T02:11:31.031 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:31 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:31.028+0000 7f3f09bd5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:11:31.031 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:31 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:31.028+0000 7f3f09bd5640 -1 osd.6 94 *** Got signal Terminated *** 2024-09-05T02:11:31.031 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:31 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:31.028+0000 7f3f09bd5640 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:11:31.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:31 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:31.082+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:31.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:31 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:31.082+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:31.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:31 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:31.082+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:31.424 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:31 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:31.410+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:31.425 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:31 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:31.410+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:31.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:31 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:31.388+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:31.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:31 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:31.388+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:31.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:31 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:31.388+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:31.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:31 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:31.722+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:31.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:31 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:31.722+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:31.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:31 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:31.588+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:31.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:31 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:31.588+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:32.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:31 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:31.764+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:32.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:31 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:31.764+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.356518+0000 front 2024-09-05T02:11:02.356330+0000 (oldest deadline 2024-09-05T02:11:27.656360+0000) 2024-09-05T02:11:32.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:32 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:32.046+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:32.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:32 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:32.046+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:32.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:32 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:32.046+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:32.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:32 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:32.393+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:32.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:32 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:32.393+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:32.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:32 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:32.393+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:32.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:32 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:32.675+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:32.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:32 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:32.675+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:32.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:32 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:32.563+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:32.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:32 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:32.563+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:32.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:32 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:32.563+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:32.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:32 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:32.419+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:32.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:32 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:32.419+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:33.148 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:32 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:32.743+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:33.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:32 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:32.743+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.356518+0000 front 2024-09-05T02:11:02.356330+0000 (oldest deadline 2024-09-05T02:11:27.656360+0000) 2024-09-05T02:11:33.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:33 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:33.023+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:33.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:33 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:33.023+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:33.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:33 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:33.023+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:33.777 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:33 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:33.407+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:33.778 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:33 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:33.407+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:33.778 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:33 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:33.407+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:33.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:33 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:33.677+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:33.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:33 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:33.677+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:33.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:33 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:33.677+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:33.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:33 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:33.608+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:33.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:33 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:33.608+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:33.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:33 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:33.608+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:33.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:33 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:33.455+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:33.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:33 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:33.455+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:34.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:33 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:33.776+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:34.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:33 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:33.776+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.356518+0000 front 2024-09-05T02:11:02.356330+0000 (oldest deadline 2024-09-05T02:11:27.656360+0000) 2024-09-05T02:11:34.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:34 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:34.017+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:34.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:34 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:34.017+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:34.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:34 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:34.017+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:34.809 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:34 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:34.455+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:34.809 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:34 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:34.455+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:34.810 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:34 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:34.455+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:34.810 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:34 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:34.455+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:34.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:34 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:34.724+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:34.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:34 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:34.724+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:34.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:34 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:34.724+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:34.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:34 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:34.577+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:34.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:34 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:34.577+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:34.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:34 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:34.577+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:34.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:34 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:34.451+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:34.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:34 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:34.451+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:35.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:34 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:34.807+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:35.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:34 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:34.807+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.356518+0000 front 2024-09-05T02:11:02.356330+0000 (oldest deadline 2024-09-05T02:11:27.656360+0000) 2024-09-05T02:11:35.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:35 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:35.020+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:35.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:35 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:35.020+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:35.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:35 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:35.020+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:35.806 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:35 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:35.436+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:35.806 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:35 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:35.436+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:35.807 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:35 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:35.436+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:35.807 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:35 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:35.436+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:35.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:35 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:35.757+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:35.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:35 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:35.757+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:35.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:35 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:35.757+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:35.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:35 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:35.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:35.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:35 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:35.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:35.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:35 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:35.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:35.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:35 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:35.496+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:35.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:35 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:35.496+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:36.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:35 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:35.804+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.754316+0000 front 2024-09-05T02:10:53.754285+0000 (oldest deadline 2024-09-05T02:11:16.654411+0000) 2024-09-05T02:11:36.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:35 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:35.804+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.356518+0000 front 2024-09-05T02:11:02.356330+0000 (oldest deadline 2024-09-05T02:11:27.656360+0000) 2024-09-05T02:11:36.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:35 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6[54949]: 2024-09-05T02:11:35.804+0000 7f3f061eb640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.957177+0000 front 2024-09-05T02:11:09.957299+0000 (oldest deadline 2024-09-05T02:11:35.158021+0000) 2024-09-05T02:11:36.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:36.039+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:36.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:36.039+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:36.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:36.039+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:36.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:36.039+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:36.881 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:36 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:36.411+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:36.881 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:36 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:36.411+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:36.881 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:36 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:36.411+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:36.881 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:36 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:36.411+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:36.881 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:36 smithi103 podman[188161]: 2024-09-05 02:11:36.577677524 +0000 UTC m=+5.669908085 container died dc9e50e94de048b7e5d154261c26e3fcece6645db134072b87f1c725e3368c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-05T02:11:36.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:36 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:36.797+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:36.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:36 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:36.797+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:36.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:36 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:36.797+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:36.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:36 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:36.617+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:36.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:36 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:36.617+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:36.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:36 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:36.617+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:36.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:36.544+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:36.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:36 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:36.544+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:37.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:37 smithi103 podman[188161]: 2024-09-05 02:11:37.098059783 +0000 UTC m=+6.190290343 container cleanup dc9e50e94de048b7e5d154261c26e3fcece6645db134072b87f1c725e3368c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-05T02:11:37.149 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:37 smithi103 bash[188161]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6 2024-09-05T02:11:37.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:37 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:37.040+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:37.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:37 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:37.040+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:37.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:37 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:37.040+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:37.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:37 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:37.040+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:37.648 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:37 smithi103 podman[188177]: 2024-09-05 02:11:37.348326199 +0000 UTC m=+0.763839160 container remove dc9e50e94de048b7e5d154261c26e3fcece6645db134072b87f1c725e3368c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-05T02:11:37.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:37 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:37.448+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:37.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:37 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:37.448+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:37.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:37 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:37.448+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:37.649 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:37 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:37.448+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:37.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:37 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:37.752+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:37.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:37 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:37.752+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:37.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:37 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:37.752+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:37.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:37 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:37.602+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:37.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:37 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:37.602+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:37.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:37 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:37.602+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:37.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:37 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:37.518+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:37.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:37 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:37.518+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:37.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:37 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:37.518+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:07.636989+0000 front 2024-09-05T02:11:07.637032+0000 (oldest deadline 2024-09-05T02:11:37.037627+0000) 2024-09-05T02:11:38.027 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:37 smithi103 podman[188268]: 2024-09-05 02:11:37.670390133 +0000 UTC m=+0.015927160 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:11:38.027 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:37 smithi103 podman[188268]: 2024-09-05 02:11:37.782547675 +0000 UTC m=+0.128084691 container create 0a96a25b9ad51a41b00f1df79206d22ac0d238c9d251a4625f6f7137d1597e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-05T02:11:38.027 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:38 smithi103 podman[188268]: 2024-09-05 02:11:38.026996436 +0000 UTC m=+0.372533451 container init 0a96a25b9ad51a41b00f1df79206d22ac0d238c9d251a4625f6f7137d1597e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-05T02:11:38.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:38 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:38.081+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:38.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:38 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:38.081+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:38.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:38 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:38.081+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:38.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:38 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:38.081+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:38.399 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:38 smithi103 podman[188268]: 2024-09-05 02:11:38.030409933 +0000 UTC m=+0.375946960 container start 0a96a25b9ad51a41b00f1df79206d22ac0d238c9d251a4625f6f7137d1597e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T02:11:38.399 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:38 smithi103 podman[188268]: 2024-09-05 02:11:38.127120498 +0000 UTC m=+0.472657537 container attach 0a96a25b9ad51a41b00f1df79206d22ac0d238c9d251a4625f6f7137d1597e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-05T02:11:38.399 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:38 smithi103 podman[188268]: 2024-09-05 02:11:38.214328038 +0000 UTC m=+0.559865073 container died 0a96a25b9ad51a41b00f1df79206d22ac0d238c9d251a4625f6f7137d1597e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True) 2024-09-05T02:11:38.881 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:38 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:38.451+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:38.882 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:38 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:38.451+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:38.882 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:38 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:38.451+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:38.882 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:38 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:38.451+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:38.896 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.6.service' 2024-09-05T02:11:38.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:38 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:38.481+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:38.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:38 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:38.481+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:38.897 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:38 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:38.481+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:07.636989+0000 front 2024-09-05T02:11:07.637032+0000 (oldest deadline 2024-09-05T02:11:37.037627+0000) 2024-09-05T02:11:38.923 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:38 smithi103 podman[188290]: 2024-09-05 02:11:38.881043725 +0000 UTC m=+0.656900526 container remove 0a96a25b9ad51a41b00f1df79206d22ac0d238c9d251a4625f6f7137d1597e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731) 2024-09-05T02:11:38.923 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:38 smithi103 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.6.service: Deactivated successfully. 2024-09-05T02:11:38.923 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:38 smithi103 systemd[1]: Stopped Ceph osd.6 for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T02:11:38.924 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 05 02:11:38 smithi103 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.6.service: Consumed 41.356s CPU time. 2024-09-05T02:11:38.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:11:38.934 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-05T02:11:38.935 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-05T02:11:38.935 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.7 2024-09-05T02:11:39.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:38 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:38.782+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:39.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:38 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:38.782+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:39.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:38 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:38.782+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:39.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:38 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:38.648+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:39.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:38 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:38.648+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:39.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:38 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:38.648+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:39.244 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:38 smithi139 systemd[1]: Stopping Ceph osd.7 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:11:39.244 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:39.127+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:39.244 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:39.127+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:39.244 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:39.127+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:39.244 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:39.127+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:39.506 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:39.242+0000 7f0528d6b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:11:39.507 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:39.242+0000 7f0528d6b640 -1 osd.7 94 *** Got signal Terminated *** 2024-09-05T02:11:39.507 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:39.242+0000 7f0528d6b640 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:11:39.507 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:39.450+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:39.507 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:39.450+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:39.507 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:39 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:39.450+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:07.636989+0000 front 2024-09-05T02:11:07.637032+0000 (oldest deadline 2024-09-05T02:11:37.037627+0000) 2024-09-05T02:11:39.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:39 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:39.428+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:39.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:39 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:39.428+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:39.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:39 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:39.428+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:39.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:39 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:39.428+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:40.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:39 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:39.746+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:40.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:39 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:39.746+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:40.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:39 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:39.746+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:40.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:39 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:39.686+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:40.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:39 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:39.686+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:40.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:39 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:39.686+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:40.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:40 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:40.137+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:40.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:40 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:40.137+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:40.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:40 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:40.137+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:40.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:40 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:40.137+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:40.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:40 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:40.433+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:40.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:40 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:40.433+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:40.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:40 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:40.433+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:07.636989+0000 front 2024-09-05T02:11:07.637032+0000 (oldest deadline 2024-09-05T02:11:37.037627+0000) 2024-09-05T02:11:40.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:40 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:40.454+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:40.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:40 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:40.454+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:40.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:40 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:40.454+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:40.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:40 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:40.454+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:41.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:40 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:40.749+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:41.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:40 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:40.749+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:41.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:40 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:40.749+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:41.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:40 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:40.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:41.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:40 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:40.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:41.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:40 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:40.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:41.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:41.100+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:41.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:41.100+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:41.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:41.100+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:41.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:41.100+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:41.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:41.461+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:41.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:41.461+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:41.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:41.461+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:07.636989+0000 front 2024-09-05T02:11:07.637032+0000 (oldest deadline 2024-09-05T02:11:37.037627+0000) 2024-09-05T02:11:41.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:41.461+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.037984+0000 front 2024-09-05T02:11:17.038003+0000 (oldest deadline 2024-09-05T02:11:41.138107+0000) 2024-09-05T02:11:41.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:41 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:41.436+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:41.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:41 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:41.436+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:41.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:41 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:41.436+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:41.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:41 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:41.436+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:42.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:41 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:41.692+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:42.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:41 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:41.692+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:42.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:41 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:41.692+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:42.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:41 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:41.777+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:42.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:41 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:41.777+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:42.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:41 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:41.777+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:42.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:41 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:41.777+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:42.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:42 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:42.117+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:42.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:42 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:42.117+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:42.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:42 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:42.117+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:42.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:42 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:42.117+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:42.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:42 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:42.469+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:42.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:42 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:42.469+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:42.897 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:42 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:42.469+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:07.636989+0000 front 2024-09-05T02:11:07.637032+0000 (oldest deadline 2024-09-05T02:11:37.037627+0000) 2024-09-05T02:11:42.897 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:42 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:42.469+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.037984+0000 front 2024-09-05T02:11:17.038003+0000 (oldest deadline 2024-09-05T02:11:41.138107+0000) 2024-09-05T02:11:42.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:42 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:42.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:42.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:42 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:42.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:42.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:42 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:42.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:42.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:42 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:42.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:43.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:42 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:42.792+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:43.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:42 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:42.792+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:43.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:42 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:42.792+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:43.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:42 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:42.792+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:43.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:42 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:42.702+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:43.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:42 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:42.702+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:43.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:42 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:42.702+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:43.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:42 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:42.702+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:43.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:43 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:43.093+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:43.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:43 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:43.093+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:43.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:43 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:43.093+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:43.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:43 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:43.093+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:43.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:43 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:43.093+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:43.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:43 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:43.478+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.034722+0000 front 2024-09-05T02:10:53.034630+0000 (oldest deadline 2024-09-05T02:11:17.734773+0000) 2024-09-05T02:11:43.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:43 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:43.478+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:01.836059+0000 front 2024-09-05T02:11:01.835986+0000 (oldest deadline 2024-09-05T02:11:24.735864+0000) 2024-09-05T02:11:43.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:43 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:43.478+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:07.636989+0000 front 2024-09-05T02:11:07.637032+0000 (oldest deadline 2024-09-05T02:11:37.037627+0000) 2024-09-05T02:11:43.896 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:43 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7[52483]: 2024-09-05T02:11:43.478+0000 7f0525381640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.037984+0000 front 2024-09-05T02:11:17.038003+0000 (oldest deadline 2024-09-05T02:11:41.138107+0000) 2024-09-05T02:11:43.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:43 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:43.435+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:43.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:43 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:43.435+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:43.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:43 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:43.435+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:43.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:43 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:43.435+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:43.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:43 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:43.435+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:44.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:43 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:43.746+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:44.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:43 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:43.746+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:44.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:43 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:43.746+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:44.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:43 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:43.746+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:44.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:43 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:43.654+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:44.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:43 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:43.654+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:44.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:43 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:43.654+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:44.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:43 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:43.654+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:44.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:44 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:44.128+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:44.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:44 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:44.128+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:44.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:44 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:44.128+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:44.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:44 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:44.128+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:44.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:44 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:44.128+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:44.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:44 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:44.444+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:44.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:44 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:44.444+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:44.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:44 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:44.444+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:44.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:44 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:44.444+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:44.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:44 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:44.444+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:45.061 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:44 smithi139 podman[69427]: 2024-09-05 02:11:44.735174677 +0000 UTC m=+5.613825138 container died 29773379962c38f001dacffc5aed4884a7178d2f249d2d588e32425361b0ff1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-05T02:11:45.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:44 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:44.742+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:45.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:44 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:44.742+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:45.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:44 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:44.742+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:45.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:44 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:44.742+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:45.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:44 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:44.664+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:45.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:44 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:44.664+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:45.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:44 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:44.664+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:45.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:44 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:44.664+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:45.370 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:45 smithi139 podman[69427]: 2024-09-05 02:11:45.159666853 +0000 UTC m=+6.038317307 container cleanup 29773379962c38f001dacffc5aed4884a7178d2f249d2d588e32425361b0ff1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-05T02:11:45.370 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:45 smithi139 bash[69427]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7 2024-09-05T02:11:45.370 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:45 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:45.148+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:45.370 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:45 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:45.148+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:45.371 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:45 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:45.148+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:45.371 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:45 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:45.148+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:45.371 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:45 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:45.148+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:45.646 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:45 smithi139 podman[69454]: 2024-09-05 02:11:45.369364707 +0000 UTC m=+0.631667412 container remove 29773379962c38f001dacffc5aed4884a7178d2f249d2d588e32425361b0ff1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-05T02:11:45.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:45 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:45.734+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:45.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:45 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:45.734+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:45.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:45 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:45.734+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:45.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:45 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:45.734+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:45.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:45 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:45.621+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:45.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:45 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:45.621+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:45.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:45 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:45.621+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:45.887 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:45 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:45.621+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:45.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:45 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:45.483+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:45.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:45 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:45.483+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:45.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:45 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:45.483+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:45.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:45 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:45.483+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:45.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:45 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:45.483+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:46.040 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:45 smithi139 podman[69542]: 2024-09-05 02:11:45.676337282 +0000 UTC m=+0.015443687 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:11:46.040 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:45 smithi139 podman[69542]: 2024-09-05 02:11:45.778523423 +0000 UTC m=+0.117629864 container create 5d19fa83a402b458e0e269b6235329c1d8efdb008dd414adc1ea6ebdcbce1ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-05T02:11:46.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:46 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:46.170+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:46.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:46 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:46.170+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:46.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:46 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:46.170+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:46.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:46 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:46.170+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:46.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:46 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:46.170+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:46.397 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:46 smithi139 podman[69542]: 2024-09-05 02:11:46.039661341 +0000 UTC m=+0.378767739 container init 5d19fa83a402b458e0e269b6235329c1d8efdb008dd414adc1ea6ebdcbce1ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-09-05T02:11:46.397 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:46 smithi139 podman[69542]: 2024-09-05 02:11:46.043480634 +0000 UTC m=+0.382587074 container start 5d19fa83a402b458e0e269b6235329c1d8efdb008dd414adc1ea6ebdcbce1ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-05T02:11:46.397 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:46 smithi139 podman[69542]: 2024-09-05 02:11:46.131116177 +0000 UTC m=+0.470222573 container attach 5d19fa83a402b458e0e269b6235329c1d8efdb008dd414adc1ea6ebdcbce1ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7-deactivate, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-05T02:11:46.397 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 05 02:11:46 smithi139 podman[69542]: 2024-09-05 02:11:46.234649309 +0000 UTC m=+0.573755712 container died 5d19fa83a402b458e0e269b6235329c1d8efdb008dd414adc1ea6ebdcbce1ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-05T02:11:46.899 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.7.service' 2024-09-05T02:11:46.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:46 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:46.450+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:46.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:46 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:46.450+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:46.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:46 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:46.450+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:46.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:46 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:46.450+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:46.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:46 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:46.450+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:46.942 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:11:46.943 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-05T02:11:46.943 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-05T02:11:46.943 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.8 2024-09-05T02:11:46.972 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:46 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:46.766+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:46.972 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:46 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:46.766+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:46.972 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:46 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:46.766+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:46.972 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:46 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:46.766+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:46.972 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:46 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:46.640+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:46.973 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:46 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:46.640+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:46.973 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:46 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:46.640+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:46.973 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:46 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:46.640+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:47.271 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:46 smithi167 systemd[1]: Stopping Ceph osd.8 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:11:47.635 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:47.270+0000 7f4a6494d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:11:47.635 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:47.270+0000 7f4a6494d640 -1 osd.8 94 *** Got signal Terminated *** 2024-09-05T02:11:47.635 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:47.270+0000 7f4a6494d640 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:11:47.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:47 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:47.177+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:47.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:47 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:47.177+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:47.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:47 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:47.177+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:47.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:47 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:47.177+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:47.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:47 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:47.177+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:47.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:47 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:47.417+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:47.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:47 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:47.417+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:47.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:47 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:47.417+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:47.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:47 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:47.417+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:47.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:47 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:47.417+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:48.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:47.798+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:48.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:47.798+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:48.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:47.798+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:48.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:47.798+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:48.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:47.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:48.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:47.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:48.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:47.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:48.137 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:47 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:47.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:48.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:48 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:48.205+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:48.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:48 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:48.205+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:48.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:48 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:48.205+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:48.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:48 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:48.205+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:48.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:48 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:48.205+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:48.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:48.426+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:48.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:48.426+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:48.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:48.426+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:48.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:48.426+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:48.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:48 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:48.426+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:49.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:48.793+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:49.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:48.793+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:49.137 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:48.793+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:49.137 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:48.793+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:49.137 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:48.708+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:49.137 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:48.708+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:49.137 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:48.708+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:49.137 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:48 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:48.708+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:49.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:49 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:49.208+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:49.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:49 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:49.208+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:49.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:49 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:49.208+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:49.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:49 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:49.208+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:49.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:49 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:49.208+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:49.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:49 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:49.453+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:49.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:49 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:49.453+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:49.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:49 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:49.453+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:49.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:49 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:49.453+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:49.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:49 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:49.453+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:49.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:49 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:49.453+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:50.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:49.792+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:50.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:49.792+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:50.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:49.792+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:50.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:49.792+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:50.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:49.721+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:50.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:49.721+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:50.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:49.721+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:50.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:49.721+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:50.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:50 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:50.214+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:50.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:50 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:50.214+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:50.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:50 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:50.214+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:50.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:50 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:50.214+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:50.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:50 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:50.214+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:50.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:50 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:50.501+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:50.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:50 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:50.501+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:50.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:50 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:50.501+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:50.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:50 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:50.501+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:50.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:50 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:50.501+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:50.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:50 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:50.501+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:51.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:50 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:50.742+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:51.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:50 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:50.742+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:51.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:50 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:50.742+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:51.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:50 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:50.742+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:51.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:50 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:50.716+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:51.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:50 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:50.716+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:51.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:50 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:50.716+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:51.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:50 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:50.716+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:51.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:51 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:51.218+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:51.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:51 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:51.218+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:51.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:51 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:51.218+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:51.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:51 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:51.218+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:51.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:51 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:51.218+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:51.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:51.541+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:51.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:51.541+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:51.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:51.541+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:51.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:51.541+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:51.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:51.541+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:51.900 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:51 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:51.541+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:52.135 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:51.717+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:53.773894+0000 front 2024-09-05T02:10:53.773751+0000 (oldest deadline 2024-09-05T02:11:16.673877+0000) 2024-09-05T02:11:52.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:51.717+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.776780+0000 front 2024-09-05T02:11:02.776656+0000 (oldest deadline 2024-09-05T02:11:28.076741+0000) 2024-09-05T02:11:52.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:51.717+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.077379+0000 front 2024-09-05T02:11:08.077306+0000 (oldest deadline 2024-09-05T02:11:33.377158+0000) 2024-09-05T02:11:52.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:51.717+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.378580+0000 front 2024-09-05T02:11:17.378483+0000 (oldest deadline 2024-09-05T02:11:40.878411+0000) 2024-09-05T02:11:52.136 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8[53818]: 2024-09-05T02:11:51.717+0000 7f4a60f63640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:26.679661+0000 front 2024-09-05T02:11:26.679694+0000 (oldest deadline 2024-09-05T02:11:51.379831+0000) 2024-09-05T02:11:52.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:51.703+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:52.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:51.703+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:52.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:51.703+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:52.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:51.703+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:52.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:51 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:51.703+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:11:52.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:52 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:52.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:52.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:52 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:52.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:52.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:52 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:52.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:52.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:52 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:52.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:52.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:52 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:52.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:52.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:52.542+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:52.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:52.542+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:52.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:52.542+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:52.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:52.542+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:52.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:52.542+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:52.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:52 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:52.542+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:52.949 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:52 smithi167 podman[71162]: 2024-09-05 02:11:52.664934537 +0000 UTC m=+5.526144848 container died 67fb60d52004d1874a96e2165cd5f9580c2289a07abe7cc5363ab59e0591b88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-05T02:11:52.949 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:52 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:52.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:52.949 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:52 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:52.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:52.949 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:52 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:52.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:52.949 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:52 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:52.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:52.950 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:52 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:52.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:11:53.323 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:53 smithi167 podman[71162]: 2024-09-05 02:11:53.046154864 +0000 UTC m=+5.907365173 container cleanup 67fb60d52004d1874a96e2165cd5f9580c2289a07abe7cc5363ab59e0591b88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-05T02:11:53.323 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:53 smithi167 bash[71162]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8 2024-09-05T02:11:53.623 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:53 smithi167 podman[71179]: 2024-09-05 02:11:53.322450803 +0000 UTC m=+0.655937586 container remove 67fb60d52004d1874a96e2165cd5f9580c2289a07abe7cc5363ab59e0591b88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T02:11:53.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:53 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:53.241+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:53.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:53 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:53.241+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:53.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:53 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:53.241+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:53.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:53 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:53.241+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:53.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:53 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:53.241+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:53.885 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:53 smithi167 podman[71270]: 2024-09-05 02:11:53.562993168 +0000 UTC m=+0.015887857 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:11:53.886 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:53 smithi167 podman[71270]: 2024-09-05 02:11:53.664917622 +0000 UTC m=+0.117812298 container create 2da213aedf8de943bbf8dc937602eab559eb68effdb06fcac15fe5a4d1013ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-05T02:11:53.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:53 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:53.622+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:53.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:53 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:53.622+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:53.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:53 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:53.622+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:53.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:53 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:53.622+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:53.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:53 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:53.622+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:11:53.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:53 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:53.496+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:53.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:53 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:53.496+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:53.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:53 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:53.496+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:53.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:53 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:53.496+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:53.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:53 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:53.496+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:53.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:53 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:53.496+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:54.151 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:53 smithi167 podman[71270]: 2024-09-05 02:11:53.942800357 +0000 UTC m=+0.395695034 container init 2da213aedf8de943bbf8dc937602eab559eb68effdb06fcac15fe5a4d1013ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8-deactivate, RELEASE=squid-91c8a7e, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-05T02:11:54.151 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:53 smithi167 podman[71270]: 2024-09-05 02:11:53.946216207 +0000 UTC m=+0.399110896 container start 2da213aedf8de943bbf8dc937602eab559eb68effdb06fcac15fe5a4d1013ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-05T02:11:54.152 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:54 smithi167 podman[71270]: 2024-09-05 02:11:54.051198474 +0000 UTC m=+0.504093156 container attach 2da213aedf8de943bbf8dc937602eab559eb68effdb06fcac15fe5a4d1013ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, ceph=True) 2024-09-05T02:11:54.404 INFO:journalctl@ceph.osd.8.smithi167.stdout:Sep 05 02:11:54 smithi167 podman[71293]: 2024-09-05 02:11:54.161814112 +0000 UTC m=+0.020776319 container died 2da213aedf8de943bbf8dc937602eab559eb68effdb06fcac15fe5a4d1013ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-05T02:11:54.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:54 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:54.222+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:54.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:54 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:54.222+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:54.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:54 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:54.222+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:54.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:54 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:54.222+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:54.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:54 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:54.222+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:54.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:54 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:54.222+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:11:54.793 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.8.service' 2024-09-05T02:11:54.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:11:54.873 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-05T02:11:54.874 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-05T02:11:54.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.9 2024-09-05T02:11:54.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:54 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:54.460+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:54.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:54 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:54.460+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:54.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:54 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:54.460+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:54.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:54 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:54.460+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:54.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:54 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:54.460+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:54.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:54 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:54.460+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:55.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:54 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:54.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:55.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:54 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:54.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:55.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:54 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:54.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:55.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:54 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:54.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:55.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:54 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:54.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:11:55.184 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:54 smithi103 systemd[1]: Stopping Ceph osd.9 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:11:55.185 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:55 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:55.182+0000 7fa582bbe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:11:55.185 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:55 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:55.182+0000 7fa582bbe640 -1 osd.9 94 *** Got signal Terminated *** 2024-09-05T02:11:55.185 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:55 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:55.182+0000 7fa582bbe640 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:11:55.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:55 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:55.224+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:55.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:55 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:55.224+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:55.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:55 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:55.224+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:55.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:55 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:55.224+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:55.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:55 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:55.224+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:55.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:55 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:55.224+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:11:55.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:55 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:55.498+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:55.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:55 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:55.498+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:55.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:55 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:55.498+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:55.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:55 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:55.498+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:55.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:55 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:55.498+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:55.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:55 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:55.498+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:56.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:55 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:55.651+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:56.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:55 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:55.651+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:56.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:55 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:55.651+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:56.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:55 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:55.651+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:56.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:55 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:55.651+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:11:56.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:56 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:56.183+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:56.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:56 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:56.183+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:56.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:56 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:56.183+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:56.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:56 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:56.183+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:56.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:56 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:56.183+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:56.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:56 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:56.183+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:11:56.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:56 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:56.462+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:56.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:56 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:56.462+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:56.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:56 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:56.462+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:56.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:56 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:56.462+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:56.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:56 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:56.462+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:56.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:56 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:56.462+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:57.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:56 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:56.648+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:57.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:56 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:56.648+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:57.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:56 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:56.648+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:57.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:56 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:56.648+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:57.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:56 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:56.648+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:11:57.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:57 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:57.190+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:57.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:57 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:57.190+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:57.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:57 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:57.190+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:57.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:57 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:57.190+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:57.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:57 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:57.190+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:57.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:57 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:57.190+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:11:57.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:57 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:57.490+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:57.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:57 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:57.490+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:57.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:57 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:57.490+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:57.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:57 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:57.490+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:57.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:57 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:57.490+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:57.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:57 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:57.491+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:58.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:57 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:57.650+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:58.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:57 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:57.650+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:58.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:57 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:57.650+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:58.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:57 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:57.650+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:58.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:57 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:57.650+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:11:58.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:58 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:58.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:58.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:58 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:58.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:58.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:58 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:58.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:58.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:58 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:58.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:58.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:58 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:58.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:58.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:58 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:58.202+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:11:58.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:58.515+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:58.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:58.515+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:58.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:58.515+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:58.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:58.515+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:58.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:58.515+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:58.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:58 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:58.515+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:11:59.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:58 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:58.642+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:59.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:58 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:58.642+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:59.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:58 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:58.642+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:59.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:58 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:58.642+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:59.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:58 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:58.642+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:11:59.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:59 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:59.249+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:11:59.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:59 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:59.249+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:11:59.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:59 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:59.249+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:11:59.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:59 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:59.249+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:11:59.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:59 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:59.249+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:11:59.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:11:59 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:11:59.249+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:11:59.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:59 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:59.597+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:11:59.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:59 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:59.597+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:11:59.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:59 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:59.597+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:11:59.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:59 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:59.597+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:11:59.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:59 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:59.597+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:11:59.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:11:59 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:11:59.597+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:11:59.898 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:59.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.406575+0000 front 2024-09-05T02:10:46.406621+0000 (oldest deadline 2024-09-05T02:11:09.906717+0000) 2024-09-05T02:11:59.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:59.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.807591+0000 front 2024-09-05T02:10:52.807816+0000 (oldest deadline 2024-09-05T02:11:16.307620+0000) 2024-09-05T02:11:59.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:59.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:02.709314+0000 front 2024-09-05T02:11:02.709211+0000 (oldest deadline 2024-09-05T02:11:28.009018+0000) 2024-09-05T02:11:59.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:59.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:08.009485+0000 front 2024-09-05T02:11:08.009524+0000 (oldest deadline 2024-09-05T02:11:33.909597+0000) 2024-09-05T02:11:59.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:59.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:19.711224+0000 front 2024-09-05T02:11:19.711350+0000 (oldest deadline 2024-09-05T02:11:42.511448+0000) 2024-09-05T02:11:59.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:11:59 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9[61668]: 2024-09-05T02:11:59.473+0000 7fa57b1cc640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.113044+0000 front 2024-09-05T02:11:27.112948+0000 (oldest deadline 2024-09-05T02:11:49.412837+0000) 2024-09-05T02:12:00.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:00 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:00.220+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:12:00.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:00 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:00.220+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:12:00.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:00 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:00.220+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:12:00.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:00 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:00.220+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:12:00.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:00 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:00.220+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:12:00.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:00 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:00.220+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:12:00.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:00 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:00.220+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.715109+0000 front 2024-09-05T02:11:33.715066+0000 (oldest deadline 2024-09-05T02:11:59.614814+0000) 2024-09-05T02:12:00.868 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:00 smithi103 podman[188382]: 2024-09-05 02:12:00.590651202 +0000 UTC m=+5.525077045 container died f4dd94a8e948c899ebe07e5a075258e79a79add2e5d5ac347bcc4cf0571f624d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2) 2024-09-05T02:12:00.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:00 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:00.588+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:00.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:00 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:00.588+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:00.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:00 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:00.588+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:00.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:00 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:00.588+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:00.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:00 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:00.588+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:00.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:00 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:00.588+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:01.149 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:01 smithi103 podman[188382]: 2024-09-05 02:12:01.033321967 +0000 UTC m=+5.967747839 container cleanup f4dd94a8e948c899ebe07e5a075258e79a79add2e5d5ac347bcc4cf0571f624d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-05T02:12:01.149 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:01 smithi103 bash[188382]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9 2024-09-05T02:12:01.617 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:01 smithi103 podman[188397]: 2024-09-05 02:12:01.201306707 +0000 UTC m=+0.607340607 container remove f4dd94a8e948c899ebe07e5a075258e79a79add2e5d5ac347bcc4cf0571f624d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-05T02:12:01.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:01 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:01.226+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:12:01.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:01 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:01.226+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:12:01.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:01 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:01.226+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:12:01.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:01 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:01.226+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:12:01.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:01 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:01.226+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:12:01.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:01 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:01.226+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:12:01.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:01 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:01.226+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.715109+0000 front 2024-09-05T02:11:33.715066+0000 (oldest deadline 2024-09-05T02:11:59.614814+0000) 2024-09-05T02:12:01.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:01.613+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:01.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:01.613+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:01.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:01.613+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:01.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:01.613+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:01.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:01.613+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:01.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:01.613+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:01.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:01 smithi103 podman[188484]: 2024-09-05 02:12:01.51965947 +0000 UTC m=+0.015792583 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:12:01.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:01 smithi103 podman[188484]: 2024-09-05 02:12:01.635415367 +0000 UTC m=+0.131548486 container create 09c33cba220640bfa49eb93250d842a2fe5ff02d672d05cca0c834cd1a5da1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T02:12:01.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:01 smithi103 podman[188484]: 2024-09-05 02:12:01.837900382 +0000 UTC m=+0.334033500 container init 09c33cba220640bfa49eb93250d842a2fe5ff02d672d05cca0c834cd1a5da1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:12:01.899 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:01 smithi103 podman[188484]: 2024-09-05 02:12:01.841383966 +0000 UTC m=+0.337517081 container start 09c33cba220640bfa49eb93250d842a2fe5ff02d672d05cca0c834cd1a5da1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9-deactivate, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-05T02:12:02.358 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:01 smithi103 podman[188484]: 2024-09-05 02:12:01.921120237 +0000 UTC m=+0.417253366 container attach 09c33cba220640bfa49eb93250d842a2fe5ff02d672d05cca0c834cd1a5da1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-05T02:12:02.358 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 05 02:12:02 smithi103 podman[188484]: 2024-09-05 02:12:02.028443665 +0000 UTC m=+0.524576788 container died 09c33cba220640bfa49eb93250d842a2fe5ff02d672d05cca0c834cd1a5da1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-05T02:12:02.642 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.9.service' 2024-09-05T02:12:02.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:02 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:02.211+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:12:02.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:02 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:02.211+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:12:02.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:02 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:02.211+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:12:02.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:02 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:02.211+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:12:02.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:02 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:02.211+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:12:02.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:02 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:02.211+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:12:02.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:02 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:02.211+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.715109+0000 front 2024-09-05T02:11:33.715066+0000 (oldest deadline 2024-09-05T02:11:59.614814+0000) 2024-09-05T02:12:02.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:02.722 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-05T02:12:02.722 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-05T02:12:02.722 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.10 2024-09-05T02:12:02.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:02 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:02.628+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:02.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:02 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:02.628+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:02.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:02 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:02.628+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:02.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:02 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:02.628+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:02.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:02 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:02.628+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:02.887 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:02 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:02.628+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:03.037 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:02 smithi139 systemd[1]: Stopping Ceph osd.10 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:03.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.035+0000 7f209c3a6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:03.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.035+0000 7f209c3a6640 -1 osd.10 94 *** Got signal Terminated *** 2024-09-05T02:12:03.396 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.035+0000 7f209c3a6640 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:12:03.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.223+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:12:03.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.223+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:12:03.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.223+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:12:03.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.223+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:12:03.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.223+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:12:03.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.223+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:12:03.397 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:03 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:03.223+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.715109+0000 front 2024-09-05T02:11:33.715066+0000 (oldest deadline 2024-09-05T02:11:59.614814+0000) 2024-09-05T02:12:03.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:03 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:03.589+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:03.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:03 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:03.589+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:03.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:03 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:03.589+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:03.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:03 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:03.589+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:03.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:03 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:03.589+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:03.887 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:03 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:03.589+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:04.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:04 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:04.193+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:12:04.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:04 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:04.193+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:12:04.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:04 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:04.193+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:12:04.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:04 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:04.193+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:12:04.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:04 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:04.193+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:12:04.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:04 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:04.193+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:12:04.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:04 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:04.193+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.715109+0000 front 2024-09-05T02:11:33.715066+0000 (oldest deadline 2024-09-05T02:11:59.614814+0000) 2024-09-05T02:12:04.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:04 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:04.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:04.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:04 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:04.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:04.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:04 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:04.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:04.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:04 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:04.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:04.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:04 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:04.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:04.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:04 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:04.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:05.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:05 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:05.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:12:05.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:05 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:05.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:12:05.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:05 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:05.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:12:05.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:05 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:05.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:12:05.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:05 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:05.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:12:05.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:05 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:05.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:12:05.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:05 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:05.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.715109+0000 front 2024-09-05T02:11:33.715066+0000 (oldest deadline 2024-09-05T02:11:59.614814+0000) 2024-09-05T02:12:05.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:05 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:05.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:05.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:05 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:05.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:05.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:05 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:05.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:05.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:05 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:05.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:05.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:05 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:05.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:05.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:05 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:05.570+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:06.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:06 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:06.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:12:06.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:06 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:06.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:12:06.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:06 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:06.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:12:06.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:06 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:06.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:12:06.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:06 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:06.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:12:06.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:06 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:06.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:12:06.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:06 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:06.230+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.715109+0000 front 2024-09-05T02:11:33.715066+0000 (oldest deadline 2024-09-05T02:11:59.614814+0000) 2024-09-05T02:12:06.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:06 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:06.614+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:06.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:06 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:06.614+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:06.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:06 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:06.614+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:06.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:06 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:06.614+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:06.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:06 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:06.614+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:06.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:06 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:06.614+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:07.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:07 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:07.251+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:43.908374+0000 front 2024-09-05T02:10:43.908404+0000 (oldest deadline 2024-09-05T02:11:08.608452+0000) 2024-09-05T02:12:07.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:07 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:07.251+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-09-05T02:10:52.709542+0000 front 2024-09-05T02:10:52.709564+0000 (oldest deadline 2024-09-05T02:11:18.009742+0000) 2024-09-05T02:12:07.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:07 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:07.251+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:00.911002+0000 front 2024-09-05T02:11:00.911110+0000 (oldest deadline 2024-09-05T02:11:24.410934+0000) 2024-09-05T02:12:07.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:07 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:07.251+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:06.712015+0000 front 2024-09-05T02:11:06.712040+0000 (oldest deadline 2024-09-05T02:11:35.512616+0000) 2024-09-05T02:12:07.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:07 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:07.251+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.213475+0000 front 2024-09-05T02:11:17.213502+0000 (oldest deadline 2024-09-05T02:11:42.513647+0000) 2024-09-05T02:12:07.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:07 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:07.251+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:22.514167+0000 front 2024-09-05T02:11:22.514092+0000 (oldest deadline 2024-09-05T02:11:53.714431+0000) 2024-09-05T02:12:07.647 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:07 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10[59423]: 2024-09-05T02:12:07.251+0000 7f20989bc640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.715109+0000 front 2024-09-05T02:11:33.715066+0000 (oldest deadline 2024-09-05T02:11:59.614814+0000) 2024-09-05T02:12:08.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:07 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:07.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:08.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:07 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:07.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:08.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:07 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:07.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:08.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:07 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:07.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:08.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:07 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:07.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:08.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:07 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:07.660+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:08.645 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:08 smithi139 podman[69646]: 2024-09-05 02:12:08.366687607 +0000 UTC m=+5.453143583 container died f21d33f614356216d92d52097ee2088ce661350b3b02b5db6a7197f10cbe59d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-05T02:12:08.896 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:08 smithi139 podman[69646]: 2024-09-05 02:12:08.727340075 +0000 UTC m=+5.813796050 container cleanup f21d33f614356216d92d52097ee2088ce661350b3b02b5db6a7197f10cbe59d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-05T02:12:08.896 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:08 smithi139 bash[69646]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10 2024-09-05T02:12:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:08 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:08.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:08 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:08.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:08 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:08.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:08 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:08.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:08 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:08.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:08 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:08.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:08 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:08.662+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-09-05T02:11:43.423228+0000 front 2024-09-05T02:11:43.423298+0000 (oldest deadline 2024-09-05T02:12:08.123129+0000) 2024-09-05T02:12:09.341 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:09 smithi139 podman[69661]: 2024-09-05 02:12:09.011696943 +0000 UTC m=+0.636465352 container remove f21d33f614356216d92d52097ee2088ce661350b3b02b5db6a7197f10cbe59d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-91c8a7e) 2024-09-05T02:12:09.646 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:09 smithi139 podman[69753]: 2024-09-05 02:12:09.243863862 +0000 UTC m=+0.016045856 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:12:09.651 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:09 smithi139 podman[69753]: 2024-09-05 02:12:09.345896714 +0000 UTC m=+0.118078699 container create 75117206bb507bfe0e49dfefb246b70b6bc7c39e7666451e777084df3566ccda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-05T02:12:09.651 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:09 smithi139 podman[69753]: 2024-09-05 02:12:09.565097115 +0000 UTC m=+0.337279108 container init 75117206bb507bfe0e49dfefb246b70b6bc7c39e7666451e777084df3566ccda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-05T02:12:09.652 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:09 smithi139 podman[69753]: 2024-09-05 02:12:09.568420915 +0000 UTC m=+0.340602902 container start 75117206bb507bfe0e49dfefb246b70b6bc7c39e7666451e777084df3566ccda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-05T02:12:09.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:09 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:09.612+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:09.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:09 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:09.612+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:09.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:09 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:09.612+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:09.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:09 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:09.612+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:09.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:09 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:09.612+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:09.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:09 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:09.612+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:09.887 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:09 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:09.612+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-09-05T02:11:43.423228+0000 front 2024-09-05T02:11:43.423298+0000 (oldest deadline 2024-09-05T02:12:08.123129+0000) 2024-09-05T02:12:10.043 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:09 smithi139 podman[69753]: 2024-09-05 02:12:09.648088252 +0000 UTC m=+0.420270245 container attach 75117206bb507bfe0e49dfefb246b70b6bc7c39e7666451e777084df3566ccda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-05T02:12:10.043 INFO:journalctl@ceph.osd.10.smithi139.stdout:Sep 05 02:12:09 smithi139 podman[69753]: 2024-09-05 02:12:09.752193374 +0000 UTC m=+0.524375358 container died 75117206bb507bfe0e49dfefb246b70b6bc7c39e7666451e777084df3566ccda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-05T02:12:10.339 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.10.service' 2024-09-05T02:12:10.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:10.381 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-05T02:12:10.381 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-05T02:12:10.381 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.11 2024-09-05T02:12:10.709 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 systemd[1]: Stopping Ceph osd.11 for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:10.710 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:10.710 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:10.710 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:10.710 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:10.710 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:10.710 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:10.710 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-09-05T02:11:43.423228+0000 front 2024-09-05T02:11:43.423298+0000 (oldest deadline 2024-09-05T02:12:08.123129+0000) 2024-09-05T02:12:10.710 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.708+0000 7f3ad3bb5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:10.710 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.708+0000 7f3ad3bb5640 -1 osd.11 94 *** Got signal Terminated *** 2024-09-05T02:12:10.711 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:10 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:10.708+0000 7f3ad3bb5640 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T02:12:11.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:11 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:11.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:11.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:11 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:11.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:11.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:11 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:11.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:11.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:11 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:11.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:11.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:11 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:11.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:11.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:11 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:11.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:11.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:11 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:11.594+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-09-05T02:11:43.423228+0000 front 2024-09-05T02:11:43.423298+0000 (oldest deadline 2024-09-05T02:12:08.123129+0000) 2024-09-05T02:12:12.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:12 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:12.619+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:12.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:12 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:12.619+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:12.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:12 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:12.619+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:12.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:12 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:12.619+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:12.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:12 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:12.619+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:12.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:12 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:12.619+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:12.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:12 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:12.619+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-09-05T02:11:43.423228+0000 front 2024-09-05T02:11:43.423298+0000 (oldest deadline 2024-09-05T02:12:08.123129+0000) 2024-09-05T02:12:13.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:13 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:13.571+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:13.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:13 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:13.571+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:13.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:13 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:13.571+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:13.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:13 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:13.571+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:13.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:13 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:13.571+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:13.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:13 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:13.571+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:13.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:13 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:13.571+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-09-05T02:11:43.423228+0000 front 2024-09-05T02:11:43.423298+0000 (oldest deadline 2024-09-05T02:12:08.123129+0000) 2024-09-05T02:12:14.841 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:14 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:14.567+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:14.841 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:14 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:14.567+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:14.842 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:14 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:14.567+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:14.842 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:14 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:14.567+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:14.842 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:14 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:14.567+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:14.842 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:14 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:14.567+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:14.842 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:14 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:14.567+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-09-05T02:11:43.423228+0000 front 2024-09-05T02:11:43.423298+0000 (oldest deadline 2024-09-05T02:12:08.123129+0000) 2024-09-05T02:12:15.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:15 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:15.599+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-05T02:10:46.815088+0000 front 2024-09-05T02:10:46.814955+0000 (oldest deadline 2024-09-05T02:11:10.314760+0000) 2024-09-05T02:12:15.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:15 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:15.599+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-09-05T02:11:03.216973+0000 front 2024-09-05T02:11:03.217027+0000 (oldest deadline 2024-09-05T02:11:27.917048+0000) 2024-09-05T02:12:15.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:15 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:15.599+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-05T02:11:09.618116+0000 front 2024-09-05T02:11:09.618083+0000 (oldest deadline 2024-09-05T02:11:32.518044+0000) 2024-09-05T02:12:15.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:15 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:15.599+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-09-05T02:11:17.819312+0000 front 2024-09-05T02:11:17.819357+0000 (oldest deadline 2024-09-05T02:11:41.919204+0000) 2024-09-05T02:12:15.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:15 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:15.599+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-09-05T02:11:27.120664+0000 front 2024-09-05T02:11:27.120689+0000 (oldest deadline 2024-09-05T02:11:51.220841+0000) 2024-09-05T02:12:15.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:15 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:15.599+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-05T02:11:33.521720+0000 front 2024-09-05T02:11:33.521662+0000 (oldest deadline 2024-09-05T02:11:59.421622+0000) 2024-09-05T02:12:15.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:15 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11[60751]: 2024-09-05T02:12:15.599+0000 7f3acc1c3640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-09-05T02:11:43.423228+0000 front 2024-09-05T02:11:43.423298+0000 (oldest deadline 2024-09-05T02:12:08.123129+0000) 2024-09-05T02:12:16.510 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:16 smithi167 podman[71384]: 2024-09-05 02:12:16.225566664 +0000 UTC m=+5.645544165 container died 198c1c2a4a35a0e074f64187e8b16f2c652ca3088179bf05b427f97647def382 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-05T02:12:16.860 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:16 smithi167 podman[71384]: 2024-09-05 02:12:16.608429389 +0000 UTC m=+6.028406888 container cleanup 198c1c2a4a35a0e074f64187e8b16f2c652ca3088179bf05b427f97647def382 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-91c8a7e) 2024-09-05T02:12:16.860 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:16 smithi167 bash[71384]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11 2024-09-05T02:12:17.135 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:16 smithi167 podman[71402]: 2024-09-05 02:12:16.859643735 +0000 UTC m=+0.628911955 container remove 198c1c2a4a35a0e074f64187e8b16f2c652ca3088179bf05b427f97647def382 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-05T02:12:17.547 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:17 smithi167 podman[71489]: 2024-09-05 02:12:17.123066645 +0000 UTC m=+0.015563305 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T02:12:17.547 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:17 smithi167 podman[71489]: 2024-09-05 02:12:17.235544711 +0000 UTC m=+0.128041407 container create 08ca77125806b76773a67fa36c8366d492ae26c496cb2a9fea91e444beb8f075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:12:17.547 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:17 smithi167 podman[71489]: 2024-09-05 02:12:17.46359728 +0000 UTC m=+0.356093945 container init 08ca77125806b76773a67fa36c8366d492ae26c496cb2a9fea91e444beb8f075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e) 2024-09-05T02:12:17.547 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:17 smithi167 podman[71489]: 2024-09-05 02:12:17.467133302 +0000 UTC m=+0.359629956 container start 08ca77125806b76773a67fa36c8366d492ae26c496cb2a9fea91e444beb8f075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-05T02:12:17.547 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:17 smithi167 podman[71489]: 2024-09-05 02:12:17.546428964 +0000 UTC m=+0.438925633 container attach 08ca77125806b76773a67fa36c8366d492ae26c496cb2a9fea91e444beb8f075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-05T02:12:17.885 INFO:journalctl@ceph.osd.11.smithi167.stdout:Sep 05 02:12:17 smithi167 podman[71489]: 2024-09-05 02:12:17.656417125 +0000 UTC m=+0.548913778 container died 08ca77125806b76773a67fa36c8366d492ae26c496cb2a9fea91e444beb8f075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-05T02:12:18.303 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@osd.11.service' 2024-09-05T02:12:18.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:18.344 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-05T02:12:18.344 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-05T02:12:18.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.a 2024-09-05T02:12:18.649 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:12:18 smithi103 systemd[1]: Stopping Ceph mds.a for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:18.904 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:12:18 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a[69086]: 2024-09-05T02:12:18.684+0000 7f37884ca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:18.905 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:12:18 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a[69086]: 2024-09-05T02:12:18.684+0000 7f37884ca640 -1 mds.a *** got signal Terminated *** 2024-09-05T02:12:21.021 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:12:20 smithi103 podman[188587]: 2024-09-05 02:12:20.744135245 +0000 UTC m=+2.205461517 container died 9a2f9e3e93e8e23a9d0ac44ff8e8143040680eed7819425ae4d2edb8c3142449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-09-05T02:12:21.378 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:12:21 smithi103 podman[188587]: 2024-09-05 02:12:21.110256186 +0000 UTC m=+2.571582457 container cleanup 9a2f9e3e93e8e23a9d0ac44ff8e8143040680eed7819425ae4d2edb8c3142449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-05T02:12:21.378 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 05 02:12:21 smithi103 bash[188587]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-a 2024-09-05T02:12:21.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.a.service' 2024-09-05T02:12:21.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:21.562 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-05T02:12:21.563 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-05T02:12:21.563 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.d 2024-09-05T02:12:21.899 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 02:12:21 smithi103 systemd[1]: Stopping Ceph mds.d for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:21.899 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 02:12:21 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d[67792]: 2024-09-05T02:12:21.863+0000 7fdb43cdc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:21.899 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 02:12:21 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d[67792]: 2024-09-05T02:12:21.863+0000 7fdb43cdc640 -1 mds.d *** got signal Terminated *** 2024-09-05T02:12:24.024 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 02:12:23 smithi103 podman[188718]: 2024-09-05 02:12:23.657380662 +0000 UTC m=+1.876818158 container died eae91d00bc66462f4ced621896da1b32f47f8335f63aaa356eff4ca0a7aa1df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T02:12:24.291 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 02:12:24 smithi103 podman[188718]: 2024-09-05 02:12:24.097682341 +0000 UTC m=+2.317119843 container cleanup eae91d00bc66462f4ced621896da1b32f47f8335f63aaa356eff4ca0a7aa1df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e) 2024-09-05T02:12:24.291 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 02:12:24 smithi103 bash[188718]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d 2024-09-05T02:12:24.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.d.service' 2024-09-05T02:12:24.557 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 02:12:24 smithi103 podman[188730]: 2024-09-05 02:12:24.290450493 +0000 UTC m=+0.626126416 container remove eae91d00bc66462f4ced621896da1b32f47f8335f63aaa356eff4ca0a7aa1df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-05T02:12:24.557 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 02:12:24 smithi103 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-05T02:12:24.557 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 05 02:12:24 smithi103 systemd[1]: Stopped Ceph mds.d for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T02:12:24.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:24.569 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-05T02:12:24.569 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-05T02:12:24.569 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.g 2024-09-05T02:12:24.894 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:12:24 smithi103 systemd[1]: Stopping Ceph mds.g for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:25.149 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:12:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-g[68676]: 2024-09-05T02:12:24.892+0000 7f38e9ef7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:25.149 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:12:24 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-g[68676]: 2024-09-05T02:12:24.892+0000 7f38e9ef7640 -1 mds.g *** got signal Terminated *** 2024-09-05T02:12:29.111 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:12:28 smithi103 podman[188840]: 2024-09-05 02:12:28.754808635 +0000 UTC m=+3.967563571 container died e264f70f4e6e3ad094fcf18b83678df35c776e8f51d4ba84f57b8ddbcd4beea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-05T02:12:29.361 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:12:29 smithi103 podman[188840]: 2024-09-05 02:12:29.209575444 +0000 UTC m=+4.422330380 container cleanup e264f70f4e6e3ad094fcf18b83678df35c776e8f51d4ba84f57b8ddbcd4beea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-05T02:12:29.361 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:12:29 smithi103 bash[188840]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-g 2024-09-05T02:12:29.612 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 05 02:12:29 smithi103 podman[188852]: 2024-09-05 02:12:29.477611509 +0000 UTC m=+0.720432776 container remove e264f70f4e6e3ad094fcf18b83678df35c776e8f51d4ba84f57b8ddbcd4beea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:12:29.634 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.g.service' 2024-09-05T02:12:29.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:29.667 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-05T02:12:29.667 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-05T02:12:29.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.j 2024-09-05T02:12:29.898 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 02:12:29 smithi103 systemd[1]: Stopping Ceph mds.j for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:30.398 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 02:12:30 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-j[67405]: 2024-09-05T02:12:30.012+0000 7ff09e896640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:30.399 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 02:12:30 smithi103 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-j[67405]: 2024-09-05T02:12:30.012+0000 7ff09e896640 -1 mds.j *** got signal Terminated *** 2024-09-05T02:12:32.173 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 02:12:31 smithi103 podman[188957]: 2024-09-05 02:12:31.909398971 +0000 UTC m=+2.017757868 container died b371690aa115daa88771e57552c8b0465b90af9858e31080fa39b228ad38232f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-05T02:12:32.540 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 02:12:32 smithi103 podman[188957]: 2024-09-05 02:12:32.280521641 +0000 UTC m=+2.388880537 container cleanup b371690aa115daa88771e57552c8b0465b90af9858e31080fa39b228ad38232f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-05T02:12:32.541 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 05 02:12:32 smithi103 bash[188957]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-j 2024-09-05T02:12:32.707 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.j.service' 2024-09-05T02:12:32.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:32.739 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-05T02:12:32.740 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-05T02:12:32.740 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.b 2024-09-05T02:12:33.070 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:12:32 smithi139 systemd[1]: Stopping Ceph mds.b for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:33.396 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:12:33 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b[65040]: 2024-09-05T02:12:33.068+0000 7ff81da06640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:33.396 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:12:33 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b[65040]: 2024-09-05T02:12:33.068+0000 7ff81da06640 -1 mds.b *** got signal Terminated *** 2024-09-05T02:12:37.044 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:12:36 smithi139 podman[69867]: 2024-09-05 02:12:36.766729631 +0000 UTC m=+3.823302436 container died 458b198b21b42a63883cdcbe8a971b9a688fa686d71da877b6cae79b7de18021 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-05T02:12:37.394 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:12:37 smithi139 podman[69867]: 2024-09-05 02:12:37.126013243 +0000 UTC m=+4.182586048 container cleanup 458b198b21b42a63883cdcbe8a971b9a688fa686d71da877b6cae79b7de18021 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:12:37.394 INFO:journalctl@ceph.mds.b.smithi139.stdout:Sep 05 02:12:37 smithi139 bash[69867]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-b 2024-09-05T02:12:37.554 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.b.service' 2024-09-05T02:12:37.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:37.590 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-05T02:12:37.590 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-05T02:12:37.591 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.e 2024-09-05T02:12:37.913 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:12:37 smithi139 systemd[1]: Stopping Ceph mds.e for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:38.396 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:12:37 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e[64436]: 2024-09-05T02:12:37.912+0000 7fae0eec7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:38.396 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:12:37 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e[64436]: 2024-09-05T02:12:37.912+0000 7fae0eec7640 -1 mds.e *** got signal Terminated *** 2024-09-05T02:12:41.048 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:12:40 smithi139 podman[69987]: 2024-09-05 02:12:40.748540427 +0000 UTC m=+2.934827479 container died e04809ef0a728af92579e0ee3550526e49b32873bbefe72bc64d1a03acb748df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-05T02:12:41.381 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:12:41 smithi139 podman[69987]: 2024-09-05 02:12:41.154387957 +0000 UTC m=+3.340675002 container cleanup e04809ef0a728af92579e0ee3550526e49b32873bbefe72bc64d1a03acb748df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, ceph=True) 2024-09-05T02:12:41.381 INFO:journalctl@ceph.mds.e.smithi139.stdout:Sep 05 02:12:41 smithi139 bash[69987]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-e 2024-09-05T02:12:41.531 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.e.service' 2024-09-05T02:12:41.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:41.565 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-05T02:12:41.565 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-05T02:12:41.565 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.h 2024-09-05T02:12:41.900 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:41 smithi139 systemd[1]: Stopping Ceph mds.h for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:42.396 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h[64739]: 2024-09-05T02:12:41.899+0000 7fe6bbdc6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:42.396 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:41 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h[64739]: 2024-09-05T02:12:41.899+0000 7fe6bbdc6640 -1 mds.h *** got signal Terminated *** 2024-09-05T02:12:45.087 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:44 smithi139 podman[70107]: 2024-09-05 02:12:44.766118906 +0000 UTC m=+2.983247242 container died 1fb4b4b48fb4a36ef73bf8675adc05f21826d46e9ca8f8a9d8777f10d110048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T02:12:45.344 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:45 smithi139 podman[70107]: 2024-09-05 02:12:45.157993902 +0000 UTC m=+3.375122241 container cleanup 1fb4b4b48fb4a36ef73bf8675adc05f21826d46e9ca8f8a9d8777f10d110048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T02:12:45.344 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:45 smithi139 bash[70107]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h 2024-09-05T02:12:45.594 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:45 smithi139 podman[70132]: 2024-09-05 02:12:45.342442692 +0000 UTC m=+0.568975380 container remove 1fb4b4b48fb4a36ef73bf8675adc05f21826d46e9ca8f8a9d8777f10d110048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-91c8a7e) 2024-09-05T02:12:45.594 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:45 smithi139 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.h.service: Deactivated successfully. 2024-09-05T02:12:45.594 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:45 smithi139 systemd[1]: Stopped Ceph mds.h for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T02:12:45.594 INFO:journalctl@ceph.mds.h.smithi139.stdout:Sep 05 02:12:45 smithi139 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.h.service: Consumed 1min 20.732s CPU time. 2024-09-05T02:12:45.596 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.h.service' 2024-09-05T02:12:45.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:45.631 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-05T02:12:45.632 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-05T02:12:45.632 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.k 2024-09-05T02:12:45.896 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:45 smithi139 systemd[1]: Stopping Ceph mds.k for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:46.397 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:45 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k[64123]: 2024-09-05T02:12:45.978+0000 7fa7f7013640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:46.397 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:45 smithi139 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k[64123]: 2024-09-05T02:12:45.978+0000 7fa7f7013640 -1 mds.k *** got signal Terminated *** 2024-09-05T02:12:49.047 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:48 smithi139 podman[70240]: 2024-09-05 02:12:48.755978137 +0000 UTC m=+2.903814808 container died 345bdedd648587d0637ce260c94ce832ae4ade219264e4df9e8cab20b580f379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T02:12:49.396 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:49 smithi139 podman[70240]: 2024-09-05 02:12:49.161378918 +0000 UTC m=+3.309215579 container cleanup 345bdedd648587d0637ce260c94ce832ae4ade219264e4df9e8cab20b580f379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e) 2024-09-05T02:12:49.396 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:49 smithi139 bash[70240]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k 2024-09-05T02:12:49.639 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.k.service' 2024-09-05T02:12:49.660 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:49 smithi139 podman[70252]: 2024-09-05 02:12:49.395836169 +0000 UTC m=+0.639935071 container remove 345bdedd648587d0637ce260c94ce832ae4ade219264e4df9e8cab20b580f379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T02:12:49.660 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:49 smithi139 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.k.service: Deactivated successfully. 2024-09-05T02:12:49.660 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:49 smithi139 systemd[1]: Stopped Ceph mds.k for 695e8e42-6b29-11ef-bcd6-c7b262605968. 2024-09-05T02:12:49.660 INFO:journalctl@ceph.mds.k.smithi139.stdout:Sep 05 02:12:49 smithi139 systemd[1]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.k.service: Consumed 3.500s CPU time. 2024-09-05T02:12:49.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:49.673 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-05T02:12:49.673 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-05T02:12:49.673 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.c 2024-09-05T02:12:49.967 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:12:49 smithi167 systemd[1]: Stopping Ceph mds.c for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:50.225 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:12:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c[66030]: 2024-09-05T02:12:49.966+0000 7f1f8b8b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:50.225 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:12:49 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c[66030]: 2024-09-05T02:12:49.966+0000 7f1f8b8b4640 -1 mds.c *** got signal Terminated *** 2024-09-05T02:12:53.069 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:12:52 smithi167 podman[71590]: 2024-09-05 02:12:52.751352689 +0000 UTC m=+2.881237987 container died 61d6142510966bf4bf5dcfa08757ab70f8df130d0ee53d4148e1e425f941ebaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-05T02:12:53.385 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:12:53 smithi167 podman[71590]: 2024-09-05 02:12:53.166828498 +0000 UTC m=+3.296713796 container cleanup 61d6142510966bf4bf5dcfa08757ab70f8df130d0ee53d4148e1e425f941ebaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-05T02:12:53.385 INFO:journalctl@ceph.mds.c.smithi167.stdout:Sep 05 02:12:53 smithi167 bash[71590]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-c 2024-09-05T02:12:53.530 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.c.service' 2024-09-05T02:12:53.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:53.567 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-05T02:12:53.567 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-05T02:12:53.567 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.f 2024-09-05T02:12:54.136 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:12:53 smithi167 systemd[1]: Stopping Ceph mds.f for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:54.136 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:12:53 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f[65731]: 2024-09-05T02:12:53.879+0000 7fb03d152640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:54.136 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:12:53 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f[65731]: 2024-09-05T02:12:53.879+0000 7fb03d152640 -1 mds.f *** got signal Terminated *** 2024-09-05T02:12:57.080 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:12:56 smithi167 podman[71724]: 2024-09-05 02:12:56.767274892 +0000 UTC m=+2.986291408 container died 044031dfc5a47254d53a18d6d32dcdbfbe5f7c431910bb53d342b9db35f93a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e) 2024-09-05T02:12:57.080 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:12:57 smithi167 podman[71724]: 2024-09-05 02:12:57.079447109 +0000 UTC m=+3.298463624 container cleanup 044031dfc5a47254d53a18d6d32dcdbfbe5f7c431910bb53d342b9db35f93a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-05T02:12:57.385 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:12:57 smithi167 bash[71724]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f 2024-09-05T02:12:57.385 INFO:journalctl@ceph.mds.f.smithi167.stdout:Sep 05 02:12:57 smithi167 podman[71735]: 2024-09-05 02:12:57.264148858 +0000 UTC m=+0.495240425 container remove 044031dfc5a47254d53a18d6d32dcdbfbe5f7c431910bb53d342b9db35f93a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-f, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T02:12:57.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.f.service' 2024-09-05T02:12:57.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:12:57.540 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-05T02:12:57.540 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-05T02:12:57.541 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.i 2024-09-05T02:12:57.876 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:12:57 smithi167 systemd[1]: Stopping Ceph mds.i for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:12:58.135 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:12:57 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i[66332]: 2024-09-05T02:12:57.874+0000 7fafff29d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:12:58.135 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:12:57 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i[66332]: 2024-09-05T02:12:57.874+0000 7fafff29d640 -1 mds.i *** got signal Terminated *** 2024-09-05T02:13:01.060 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:13:00 smithi167 podman[71841]: 2024-09-05 02:13:00.764927689 +0000 UTC m=+3.003115652 container died 6155c037fa44f7d63023dc796e53497ff3c925a241e2fb9b552f76e2e60673d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T02:13:01.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:13:01 smithi167 podman[71841]: 2024-09-05 02:13:01.141463643 +0000 UTC m=+3.379651595 container cleanup 6155c037fa44f7d63023dc796e53497ff3c925a241e2fb9b552f76e2e60673d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e) 2024-09-05T02:13:01.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Sep 05 02:13:01 smithi167 bash[71841]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-i 2024-09-05T02:13:01.561 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.i.service' 2024-09-05T02:13:01.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:13:01.595 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-05T02:13:01.595 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-05T02:13:01.595 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.l 2024-09-05T02:13:01.913 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:13:01 smithi167 systemd[1]: Stopping Ceph mds.l for 695e8e42-6b29-11ef-bcd6-c7b262605968... 2024-09-05T02:13:02.385 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:13:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l[65425]: 2024-09-05T02:13:01.912+0000 7fde83db8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T02:13:02.385 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:13:01 smithi167 ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l[65425]: 2024-09-05T02:13:01.912+0000 7fde83db8640 -1 mds.l *** got signal Terminated *** 2024-09-05T02:13:05.022 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:13:04 smithi167 podman[71965]: 2024-09-05 02:13:04.759018317 +0000 UTC m=+2.951194066 container died b511a52653528d21bfa3c0bd19a811094eba3bbbcbd18aea81a0c25b54563177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-05T02:13:05.363 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:13:05 smithi167 podman[71965]: 2024-09-05 02:13:05.12055634 +0000 UTC m=+3.312732088 container cleanup b511a52653528d21bfa3c0bd19a811094eba3bbbcbd18aea81a0c25b54563177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-05T02:13:05.364 INFO:journalctl@ceph.mds.l.smithi167.stdout:Sep 05 02:13:05 smithi167 bash[71965]: ceph-695e8e42-6b29-11ef-bcd6-c7b262605968-mds-l 2024-09-05T02:13:05.486 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-695e8e42-6b29-11ef-bcd6-c7b262605968@mds.l.service' 2024-09-05T02:13:05.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T02:13:05.518 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-05T02:13:05.519 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T02:13:05.720 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T02:13:08.151 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T02:13:08.352 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T02:13:10.746 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T02:13:10.948 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T02:13:13.341 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T02:13:13.375 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T02:13:13.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T02:13:13.437 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-05T02:13:13.439 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793/remote/smithi103/crash 2024-09-05T02:13:13.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T02:13:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T02:13:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-09-05T02:13:13.471 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793/remote/smithi139/crash 2024-09-05T02:13:13.472 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T02:13:13.508 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T02:13:13.509 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now 2024-09-05T02:13:13.511 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793/remote/smithi167/crash 2024-09-05T02:13:13.511 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T02:13:13.557 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T02:13:13.557 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2024-09-05T02:13:13.558 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-05T02:13:13.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-05T02:13:13.598 INFO:tasks.cephadm:Compressing logs... 2024-09-05T02:13:13.598 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T02:13:13.642 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T02:13:13.645 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T02:13:13.666 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-05T02:13:13.666 INFO:teuthology.orchestra.run.smithi103.stderr:: No such file or directory 2024-09-05T02:13:13.666 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.0.log 2024-09-05T02:13:13.668 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.9.log 2024-09-05T02:13:13.668 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T02:13:13.669 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T02:13:13.669 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.a.log 2024-09-05T02:13:13.669 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-volume.log: /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T02:13:13.669 INFO:teuthology.orchestra.run.smithi103.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-05T02:13:13.669 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mon.a.log 2024-09-05T02:13:13.675 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.g.log 2024-09-05T02:13:13.675 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mon.a.log: 82.7% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T02:13:13.675 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.6.log 2024-09-05T02:13:13.675 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T02:13:13.675 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.g.log: /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T02:13:13.675 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.d.log 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log: 86.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mgr.x.log 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.10.log 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi139.stderr:: No such file or directory 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.1.log 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.7.log 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.b.log 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.k.log 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-volume.log: /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.4.log 2024-09-05T02:13:13.676 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T02:13:13.679 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T02:13:13.679 INFO:teuthology.orchestra.run.smithi167.stderr:: No such file or directory 2024-09-05T02:13:13.679 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T02:13:13.679 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.i.log 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mon.c.log 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.11.log 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mon.c.log: /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.2.log 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr: 82.7% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.5.log 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.l.log 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr: 90.5% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.8.log 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.l.log: /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log: 86.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T02:13:13.680 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mgr.z.log 2024-09-05T02:13:13.682 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.d.log.gz 2024-09-05T02:13:13.682 INFO:teuthology.orchestra.run.smithi103.stderr: 91.0% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T02:13:13.682 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.j.log 2024-09-05T02:13:13.682 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.3.log 2024-09-05T02:13:13.683 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T02:13:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.4.log: 91.0% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T02:13:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T02:13:13.685 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.c.log 2024-09-05T02:13:13.686 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.j.log: 94.8% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.j.log.gz 2024-09-05T02:13:13.687 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-05T02:13:13.687 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T02:13:13.691 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mgr.z.log: 92.8% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mgr.z.log.gz 2024-09-05T02:13:13.692 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.f.log 2024-09-05T02:13:13.694 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mgr.y.log 2024-09-05T02:13:13.694 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T02:13:13.695 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T02:13:13.696 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T02:13:13.701 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mgr.y.log: 92.7% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mgr.y.log.gz 2024-09-05T02:13:13.705 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T02:13:13.706 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.h.log 2024-09-05T02:13:13.710 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log: 86.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T02:13:13.710 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mon.b.log 2024-09-05T02:13:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.e.log 2024-09-05T02:13:13.726 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T02:13:13.727 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T02:13:13.735 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T02:13:13.759 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T02:13:13.926 INFO:teuthology.orchestra.run.smithi103.stderr: 90.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mgr.x.log.gz 2024-09-05T02:13:14.233 INFO:teuthology.orchestra.run.smithi167.stderr: 91.8% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mon.c.log.gz 2024-09-05T02:13:14.345 INFO:teuthology.orchestra.run.smithi139.stderr: 91.9% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mon.b.log.gz 2024-09-05T02:13:14.658 INFO:teuthology.orchestra.run.smithi139.stderr: 94.6% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.k.log.gz 2024-09-05T02:13:14.725 INFO:teuthology.orchestra.run.smithi103.stderr: 90.5% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mon.a.log.gz 2024-09-05T02:13:14.933 INFO:teuthology.orchestra.run.smithi103.stderr: 94.5% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.a.log.gz 2024-09-05T02:13:15.295 INFO:teuthology.orchestra.run.smithi103.stderr: 94.2% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.g.log.gz 2024-09-05T02:13:15.397 INFO:teuthology.orchestra.run.smithi167.stderr: 94.5% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.f.log.gz 2024-09-05T02:13:18.377 INFO:teuthology.orchestra.run.smithi167.stderr: 94.5% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.l.log.gz 2024-09-05T02:13:20.625 INFO:teuthology.orchestra.run.smithi103.stderr: 91.9% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.0.log.gz 2024-09-05T02:13:21.898 INFO:teuthology.orchestra.run.smithi103.stderr: 91.6% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.3.log.gz 2024-09-05T02:13:21.985 INFO:teuthology.orchestra.run.smithi139.stderr: 91.7% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.10.log.gz 2024-09-05T02:13:22.004 INFO:teuthology.orchestra.run.smithi167.stderr: 91.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.8.log.gz 2024-09-05T02:13:22.903 INFO:teuthology.orchestra.run.smithi139.stderr: 92.0% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.1.log.gz 2024-09-05T02:13:23.176 INFO:teuthology.orchestra.run.smithi103.stderr: 91.4% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.6.log.gz 2024-09-05T02:13:23.281 INFO:teuthology.orchestra.run.smithi167.stderr: 91.5% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.5.log.gz 2024-09-05T02:13:23.319 INFO:teuthology.orchestra.run.smithi103.stderr: 91.5% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.9.log.gz 2024-09-05T02:13:23.540 INFO:teuthology.orchestra.run.smithi167.stderr: 91.6% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.11.log.gz 2024-09-05T02:13:23.954 INFO:teuthology.orchestra.run.smithi139.stderr: 91.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.7.log.gz 2024-09-05T02:13:24.450 INFO:teuthology.orchestra.run.smithi167.stderr: 91.5% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.2.log.gz 2024-09-05T02:13:25.544 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-osd.4.log.gz 2024-09-05T02:13:38.246 INFO:teuthology.orchestra.run.smithi167.stderr: 94.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.c.log.gz 2024-09-05T02:13:39.399 INFO:teuthology.orchestra.run.smithi139.stderr: 94.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.h.log.gz 2024-09-05T02:13:45.345 INFO:teuthology.orchestra.run.smithi139.stderr: 94.4% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.b.log.gz 2024-09-05T02:13:49.607 INFO:teuthology.orchestra.run.smithi139.stderr: 94.3% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.e.log.gz 2024-09-05T02:13:49.609 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-05T02:13:49.609 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m35.950s 2024-09-05T02:13:49.609 INFO:teuthology.orchestra.run.smithi139.stderr:user 2m8.612s 2024-09-05T02:13:49.609 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m4.094s 2024-09-05T02:14:22.063 INFO:teuthology.orchestra.run.smithi167.stderr: 93.8% -- replaced with /var/log/ceph/695e8e42-6b29-11ef-bcd6-c7b262605968/ceph-mds.i.log.gz 2024-09-05T02:14:22.064 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-05T02:14:22.064 INFO:teuthology.orchestra.run.smithi167.stderr:real 1m8.405s 2024-09-05T02:14:22.065 INFO:teuthology.orchestra.run.smithi167.stderr:user 2m13.553s 2024-09-05T02:14:22.065 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m4.173s 2024-09-05T02:15:12.312 INFO:teuthology.orchestra.run.smithi103.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-05T02:15:12.313 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-05T02:15:12.313 INFO:teuthology.orchestra.run.smithi103.stderr:real 1m58.657s 2024-09-05T02:15:12.313 INFO:teuthology.orchestra.run.smithi103.stderr:user 2m31.443s 2024-09-05T02:15:12.313 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m4.682s 2024-09-05T02:15:12.314 INFO:tasks.cephadm:Archiving logs... 2024-09-05T02:15:12.314 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793/remote/smithi103/log 2024-09-05T02:15:12.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T02:15:37.730 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793/remote/smithi139/log 2024-09-05T02:15:37.731 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T02:15:54.566 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793/remote/smithi167/log 2024-09-05T02:15:54.567 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T02:16:13.003 INFO:tasks.cephadm:Removing cluster... 2024-09-05T02:16:13.003 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 --force 2024-09-05T02:16:13.206 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T02:16:13.587 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 --force 2024-09-05T02:16:13.792 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T02:16:14.187 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 695e8e42-6b29-11ef-bcd6-c7b262605968 --force 2024-09-05T02:16:14.388 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 695e8e42-6b29-11ef-bcd6-c7b262605968 2024-09-05T02:16:14.796 INFO:tasks.cephadm:Removing cephadm ... 2024-09-05T02:16:14.797 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T02:16:14.816 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T02:16:14.833 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T02:16:14.849 INFO:tasks.cephadm:Teardown complete 2024-09-05T02:16:14.849 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-05T02:16:14.860 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-05T02:16:14.861 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T02:16:14.863 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T02:16:14.875 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T02:16:14.986 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-05T02:16:14.996 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-05T02:16:14.997 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T02:16:15.000 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T02:16:15.002 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T02:16:15.013 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-05T02:16:15.014 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-05T02:16:15.016 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-09-05T02:16:15.017 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T02:16:15.017 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-05T02:16:15.018 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 +238us[ +249us] +/- 49ms 2024-09-05T02:16:15.018 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -547us[ -547us] +/- 64ms 2024-09-05T02:16:15.018 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +408us[ +408us] +/- 90ms 2024-09-05T02:16:15.018 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T02:16:15.019 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T02:16:15.019 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-05T02:16:15.019 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 +233us[ +253us] +/- 49ms 2024-09-05T02:16:15.019 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 -565us[ -546us] +/- 64ms 2024-09-05T02:16:15.019 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +397us[ +416us] +/- 90ms 2024-09-05T02:16:15.020 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T02:16:15.020 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T02:16:15.020 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-09-05T02:16:15.020 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +221us[ +231us] +/- 49ms 2024-09-05T02:16:15.020 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -547us[ -547us] +/- 64ms 2024-09-05T02:16:15.020 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +416us[ +416us] +/- 90ms 2024-09-05T02:16:15.020 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T02:16:15.021 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-05T02:16:15.057 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-05T02:16:15.057 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-05T02:16:15.068 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T02:16:15.147 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T02:16:15.213 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T02:16:15.280 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T02:16:15.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T02:16:15.309 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T02:16:15.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T02:16:15.337 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T02:16:15.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T02:16:15.366 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-05T02:16:15.378 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-05T02:16:15.390 INFO:teuthology.task.internal:Duration was 1800.696843 seconds 2024-09-05T02:16:15.390 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-05T02:16:15.400 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-05T02:16:15.400 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T02:16:15.403 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T02:16:15.405 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T02:16:15.440 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T02:16:15.442 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T02:16:15.444 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T02:16:15.937 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-05T02:16:15.937 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-09-05T02:16:15.938 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T02:16:15.958 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-09-05T02:16:15.958 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T02:16:16.009 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-09-05T02:16:16.010 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T02:16:16.038 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-05T02:16:16.039 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T02:16:16.041 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T02:16:16.052 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T02:16:16.106 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-05T02:16:16.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T02:16:16.109 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T02:16:16.119 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T02:16:16.795 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-05T02:16:16.806 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-05T02:16:16.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T02:16:16.831 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T02:16:16.858 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T02:16:16.884 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-05T02:16:16.902 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T02:16:16.905 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T02:16:16.908 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T02:16:16.927 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-09-05T02:16:16.929 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-09-05T02:16:16.947 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-09-05T02:16:16.998 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T02:16:17.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T02:16:17.013 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T02:16:17.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T02:16:17.041 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T02:16:17.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T02:16:17.056 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-05T02:16:17.068 INFO:teuthology.task.internal:Transferring archived files... 2024-09-05T02:16:17.069 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793/remote/smithi103 2024-09-05T02:16:17.070 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T02:16:17.120 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793/remote/smithi139 2024-09-05T02:16:17.121 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T02:16:17.158 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889793/remote/smithi167 2024-09-05T02:16:17.158 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T02:16:17.198 INFO:teuthology.task.internal:Removing archive directory... 2024-09-05T02:16:17.198 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T02:16:17.201 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T02:16:17.203 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T02:16:17.247 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-05T02:16:17.266 INFO:teuthology.task.internal:Not uploading archives. 2024-09-05T02:16:17.266 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-05T02:16:17.278 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-05T02:16:17.278 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T02:16:17.281 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T02:16:17.283 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T02:16:17.293 INFO:teuthology.orchestra.run.smithi103.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 02:16 /home/ubuntu/cephtest 2024-09-05T02:16:17.295 INFO:teuthology.orchestra.run.smithi139.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 02:16 /home/ubuntu/cephtest 2024-09-05T02:16:17.303 INFO:teuthology.orchestra.run.smithi167.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 02:16 /home/ubuntu/cephtest 2024-09-05T02:16:17.304 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-05T02:16:17.316 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-05T02:16:17.343 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 1800.6968433856964 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-05T02:16:17.343 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T02:16:17.421 INFO:teuthology.run:pass